gulp.src globbing: re-include files that were excluded by globs?

[UltCombo]

In gulp.src, is it at all possible to re-include files that were excluded by a previous glob? This is what I mean:

gulp.task('default', function() {
    rimraf.sync('dist');
    return gulp.src(['src/**', '!src/**/*.js', 'src/vendor/**/*.js'], { base: 'src' })
        .pipe(gulp.dest('dist'));
});

This results in an empty dist/vendor directory.

[heikki]

No, not like that at least. Negations (globs starting with !) are always done last regardless of the order they were specified.

[UltCombo]

Thanks for the explanation.
However, this behavior was unexpected for me. multimatch, for example, respects the glob patterns order.

In my use case, a globs array similar to the one described in the OP's test case is generated automatically from the negation of other globs, so the order is important to me.

Is this a design choice? A limitation of a dependency? Something worth of merging if I propose a PR? Or at least worth documenting?

[UltCombo]

In worst case, guess I can always fork vinyl-fs.

[UltCombo]

I guess it would be possible to workaround this issue with:

return gulp.src(['src/**', '!src/**/*.js'])
    .pipe(gulp.src(['src/vendor/**/*.js']))
    // ...

But such a stream would be a pain to produce dynamically (even more so because a later negated glob could affect an earlier gulp.src and the approach above does not cover that use case). Perhaps applying gulp-filter for the negations would be the best here:

return gulp.src(['src/**'])
    .pipe(filter(['!src/**/*.js']))
    .pipe(gulp.src(['src/vendor/**/*.js']))
    // ...

[heikki]

This recent StackOverflow question looks similar: http://stackoverflow.com/questions/27622702/gulpjs-negate-file-then-re-add-it-later

[sindresorhus]

Yeah that's unexpected indeed. Unless there a good technical reason it's like that it should be fixed.

[heikki]

All negative globs apply to the entire stream, see: https://github.com/wearefractal/glob-stream/blob/master/index.js#L79-L85
This is the correct way to do things. What we are discussing now should be documentation of this choice to prevent problems as described in the original post.

https://github.com/gulpjs/gulp/issues/428#issuecomment-41622255

[UltCombo]

Thanks @heikki, I'll close this as a dupe of your linked issue.
Although, the linked issue discusses both globs order and file emission order, while I'm only concerned with the globs order.

[UltCombo]

I just realized gulp.src does not return a pass-through stream, so my samples above do not work.
How come something seemingly so simple becomes so damn complicated.

[UltCombo]

FWIMC, I've made a gulp.src wrapper which respects the globs order: gulp-src-ordered-globs

[heikki]

Adding stuff to stream has an open issue: https://github.com/wearefractal/vinyl-fs/issues/25

[contra]

@UltCombo Why don't you send a PR to add fixes to vinyl-fs instead of forking it and publishing a new module? If you fix the passthrough issue (it is an open bug) or add the ability to respect ordering without harming perf I would definitely be open to merging that...

I'm driving a uhaul across the country right now and haven't had time to look at this - but please don't get so impatient that you fork

[UltCombo]

@contra oh sorry, forking was the first thing that came to my mind, but soon I realized it was a bad idea and completely unnecessary. If you look at the gulp-src-ordered-globs source code, it just wraps gulp.src() into merge-stream and gulp-filter to achieve the desired behavior. The logic is a bit convoluted, but it works.

Ordered globs are an essential part of the workflow that I'm developing, that's why I made gulp-src-ordered-globs as a temporary workaround for this issue until it is properly fixed (or as a definitive workaround in case you wouldn't want to fix this ;)).

I'll take a look and open a PR in glob-stream to respect globs order then.

[UltCombo]

Proposed wearefractal/glob-stream#27
I've flattened out my filtering logic so it won't create any extra stream. It shouldn't harm the perf now.

By the way, as this is technically a breaking change, will we have to wait until gulp 4 to land this on gulp? You could consider the current orderless behavior a bug/misuse, but even then this fix would fall into the breaking change category. If so, I'll keep gulp-src-ordered-globs around until gulp 4 is released.

[contra]

@UltCombo Yeah it is a breaking change, will go into 4.0

[UltCombo]

Alright, I've added a note to the top of gulp-src-ordered-globs's readme about gulp 4, and I'll mark the npm package as deprecated when gulp 4 stable is released.

I didn't intend to appear impatient, but I really needed this functionality to make a decent gulp workflow for me and the rest of @es6rocks to work on our projects -- that's why I had to create another package. :wink:

[caroso1222]

+1 for @UltCombo's gulp-src-ordered-globs package. It does the work just perfect.