gulp.watch with **/*.js and multiple globs not working on Windows

[adamreisnz]

When watching files with Gulp 4, I have the following watch:

gulp.watch(['app/**/*.js', 'app/components/**/*.html']);

On OS X, it works fine for both JS and HTML files, but on Windows 7, making a file change to a JS file does not get detected.

Changing this to the following fixes the issue:

gulp.watch(['app/**/*']);

Also, watching _only_ JS files also works as intended:

gulp.watch(['app/**/*.js']);

It seems only the combination of both globs breaks something causing JS files not to be watched on windows.

Possibly related to https://github.com/floatdrop/gulp-watch/issues/194

[contra]

Can you try gulp 4.0 which has a new file watcher that fixes these bugs?

[adamreisnz]

@contra This is with Gulp 4.

[contra]

@adambuczynski Are you able to reproduce these issues using chokidar directly? If so, open the issue there and LMK

[adamreisnz]

Will check. Problem is I don't have access to the windows computer directly, as I was helping someone else getting the build process up and running on their windows machine remotely. However, I will try to reproduce with chokidar directly.

[adamreisnz]

@contra Have tested with a simple example, just chokidar v1.4.2:

var chokidar = require('chokidar');
chokidar.watch(['app/**/*.js', 'app/components/**/*.html']).on('all', (event, path) => {
  console.log(event, path);
});

Folder structure:

Result is that change detection _does_ work with just chokidar, for both HTML and JS files:

So it seems to be a gulp issue (unless gulp uses an older version of chokidar which was bugged)

[phated]

Please test "change", "unlink" and "add" events, as those are the chokidar events we listen to.

[adamreisnz]

As I said, I don't have direct access to the Windows computer, and I feel I've already exhausted them with asking them to test it with just Chokidar.

I've taken a look at the Gulp 4 source code and I don't see anything that could cause the issue, so I'm at a bit of loss here. Using chokidar directly seems to work fine.

I'll try to devise another isolated test case with both gulp and chokidar and will report back once I know more.

[adamreisnz]

Is anyone else with access to a windows 7 computer able to reproduce this?

[thasmo]

I could test this on Windows 10. May I ask you to provide a full example of how you do it with Gulp 4? Thanks!

[adamreisnz]

@thasmo you can find a "full" example in this Angular 1 seed project: https://github.com/meanie/angular-seed

Should be good to go out of the box after npm install. Run npm start to fire up the dev environment, which will run the gulp watch as well as serve the client via Browser sync. If you run into any issues let me know. I haven't done extensive testing of it on Windows, but someone else is using it on Windows 7 and it was running ok.

The watch task itself can be found in build/tasks/watch-app-js.js, and the files that are watched is configurable in build/config.js

[thasmo]

Tried it and seems like app/**/*.js gets ignored i.e. not watched, although I didn't have a closer look because the project's build setup is rather complex. Could it be that app/components/**/*.html somehow overwrites or invlidates app/**/*.js?

[adamreisnz]

It shouldn't, it's been for me, but I'll do a fresh install when I'm back home and have another look.

The constants for the files are both arrays, and they get merged in the watch task, which then gets passed as an array to gulp.watch;

let files = [].concat(APP_JS_SRC, APP_HTML_SRC);
gulp.watch(files, debounce(
  gulp.series(lint, buildAppJs, buildIndex), WATCH_DEBOUNCE_DELAY
));

[adamreisnz]

I've installed the angular seed and tested, and on OS X the watches are working correctly. So in your case @thasmo, when you run the project and make changes to HTML files in the app/components folder, does the watch get triggered? How about for js files?

[thasmo]

That's what I meant. Changing HTML gets triggered correctly, JS changes are not picked up.

[adamreisnz]

Ok so that basically that confirms that you can replicate the issue we've experience on Windows 10 as well. What if you create this simple example with chokidar only? https://github.com/gulpjs/gulp/issues/1491#issuecomment-172689471

Could it be that creating multiple watchers is what's affecting chokidar? Maybe if another watcher negates files listed in a previous watcher it somehow overrides / merges the rules? I'll create another isolated test case for that.

[adamreisnz]

@thasmo could you check with these two test scripts on windows and see what console output you get when you edit the test.js and test.html files?

Folder/file structure:

package.json:

{
  "name": "chokidar-gulp-test",
  "version": "1.0.0",
  "description": "Simple test",
  "author": "Adam Buczynski",
  "license": "ISC",
  "dependencies": {
    "chokidar": "^1.4.2",
    "gulp": "gulpjs/gulp#4.0"
  }
}

chokidar.js:

'use strict';

const APP_JS_SRC = ['app/**/*.js'];
const APP_HTML_SRC = ['app/components/**/*.html'];
const LIB_JS_SRC = ['lib/**/*.js'];

let appFiles = [].concat(APP_JS_SRC, APP_HTML_SRC);
let libFiles = LIB_JS_SRC;

let chokidar = require('chokidar');
chokidar.watch(appFiles).on('all', (event, path) => {
  console.log('app file:', event, path);
});
chokidar.watch(libFiles).on('all', (event, path) => {
  console.log('lib file:', event, path);
});

gulp.js

'use strict';

const APP_JS_SRC = ['app/**/*.js'];
const APP_HTML_SRC = ['app/components/**/*.html'];
const LIB_JS_SRC = ['lib/**/*.js'];

let appFiles = [].concat(APP_JS_SRC, APP_HTML_SRC);
let libFiles = LIB_JS_SRC;

function doAppStuff() {
  console.log('app file changed');
}
function doLibStuff() {
  console.log('lib file changed');
}

let gulp = require('gulp');
gulp.watch(appFiles, doAppStuff);
gulp.watch(libFiles, doLibStuff);

Then test both separately with:

node chokidar.js
node gulp.js

Do the console logs get triggered properly whenever you make file changes for both the chokidar and gulp scripts?

On OS X, both work as expected for me. If they both work for you on Windows, there might be something else that's messing things up in the Angular seed project, but I can't see what that would be.

[replete]

I've had some weird issues which were all down to missing Windows packages. Try this:

• Ensure node-gyp has no errors with missing headers/libs
  
  
  
  1. Open File > New > Project in VS2015
  
  
  2. Browse Other Languages > C++
  
  
  3. Select Install Visual C++ 2015 tools for Windows Desktop, hit OK, install (Adds Visual C++ libs and Windows 8.1 SDK)
  
  

(You'll have to find these two packages manually if you don't have Visual Studio 2015)

[RobVanGroenewoud]

@adambuczynski: I just checked your proposed code on Windows 7.
Both chokidar and gulp are receiving the events properly for both file types.

fwiw: I do have Visual Studio 2015 installed.

[adamreisnz]

@RobVanGroenewoud thanks for checking, I will try to do some more debugging on the specific machine where it's failing.

[puranjayjain]

@RobVanGroenewoud I'm also facing he same issue as yours but on windows 10 x64, @adambuczynski unfortunately your code didnt work out for me here :sob: any suggestions anyone?

[adamreisnz]

@puranjayjain sorry, not sure how to debug this, as I don't have access to windows. And quite frankly, since I only work on Mac myself, this issue doesn't impact me much so I don't want to spend too much more time on it.

If this comes back to bite me again some time I might spend some more time analysing it, but for now I wouldn't know what to do about it.

[jimmythomson]

@adambuczynski - your test code (https://github.com/gulpjs/gulp/issues/1491#issuecomment-184439594) all works fine for me on Windows 7 Enterprise SP1, however, your angular-seed app does not pick up JS changes inside the app/components folder.

[phated]

Copying @es128 here. Maybe he has some insights

[adamreisnz]

@phated I have now started to experience this exact same problem on OS X 10.11.2.
With the following watch in place:

gulp.watch(['app/**/*.js', 'app/components/**/*.html']);

If I make changes to a .js file in app/components gulp watch is _not_ triggered.
If I make changes to a .html file in app/components _or_ changes to a .js file in app/somethingElse, gulp watch is triggered.

It seems that the glob for .html files is somehow preventing .js files in the same folder from being watched.

If I explicitly specify the JS files in the components folder as well, or if I take out the .html glob:

gulp.watch(['app/**/*.js', 'app/components/**/*.js', 'app/components/**/*.html']);
//or
gulp.watch(['app/**/*.js']);

It works as expected.

Not sure if this is a gulp-watch problem or an issue with the underlying glob-watcher.

I will to create an isolated test case.

[phated]

@adambuczynski if you are using gulp 4 with the function syntax, you probably aren't using a proper async function, which is a different issue (many duplicates).

[adamreisnz]

@phated not sure what you mean, I have been using the same code for months now, and it's always worked well.

let files = [].concat(APP_JS_SRC, APP_HTML_SRC);
  gulp.watch(files, debounce(
    gulp.series(lint, buildAppJs, buildIndex), WATCH_DEBOUNCE_DELAY
  ));

The debounce is in there because at the time you hadn't included that in glob-watcher yet. If that's been updated I can probably remove those now. I'm using [email protected].

[phated]

@adambuczynski that's because debounce functionality wasn't introduced into gulp 4 yet. I'm guessing you just updated. You watch functions must signal async completion now. The docs were written this way for a long time but no one was writing code this way but there were subtle bugs that allowed it to work. This functionality has been normalized now (tasks & watch require the same async completion).

[adamreisnz]

@phated Yes, I recently updated gulp. Does that mean I can get rid of the manual debouncing now?
I'll remove them and double check the task functions.

[phated]

@adambuczynski yeah, you can remove the manual debouncing and if you keep the gulp.series call, everything should just work because it returns an async function.

[adamreisnz]

@phated Unfortunately, it doesn't seem to work. Still has the same problem.

Setup is now:

let files = [].concat(APP_JS_SRC, APP_HTML_SRC);
gulp.watch(files, gulp.series(lint, buildAppJs, buildIndex));

Only when I explicitly add app/components/**/*.js to the globs does it detect file changes in those JS files. If I leave that bit out and only watch app/**/*.js it doesn't detect them.

I will make an isolated test case.

[WestonThayer]

+1, I repro this on Windows 10, x64. Any way I can help?

[adamreisnz]

I have been unsuccessful so far in creating an isolated test case and just added the sub directory to my globs to get around this issue. However @WestonThayer if you are also experiencing this, maybe we can figure out together what's causing it.

Have you been able to reproduce this consistently with an isolated test case?

[WestonThayer]

I debugged it a bit, turns out that it's not actually hung, gaze.js is just taking a long time to watch ~2,000 files. I got tricked because the first glob I tested with only turns up 10 files, thus it didn't hang with just that one. Then I added the second glob, which targeted close to 2,000, so that caused the hang.

My scenario is to watch all asset-type files (.png, .jpg, etc) for updates, then copy them to my build/ directory. I do have thousands. Is there a better way to go about this?

[contra]

@WestonThayer Use gulp 4.0 which has a better globbing engine (chokidar)

[WestonThayer]

I see that's not on npm yet. What's the best way to consume it?

[adamreisnz]

@WestonThayer

https://demisx.github.io/gulp4/2015/01/15/install-gulp4.html

For example usage, see this Gulpfile and related tasks (in build/tasks).

[phated]

I'm going to be closing this as it doesn't seem to be reproducible. We hand everything off to chokidar, so if it crops up again, it will probably need to opened over there.

[adamreisnz]

Well, I have given up on trying to debug this and resorted to just specifying the components path in the folders to watch. It's not pretty, but at least it works :(

It's also worth noting that for some new Angular 2 projects we've been migrating away from Gulp to Webpack, and haven't been experiencing issues like these there yet. I believe Webpack also uses Chokidar to watch for file changes.

[phated]

@adambuczynski webpack doesn't operate on globs :/

[mmfilesi]

Thats work (window 10, browserSync).

gulp.task('build', gulp.parallel('prepareScripts', 'prepareStatics', (done)=> {
    done();
}));
gulp.task('server', ()=> {
  browserSync.init(optionsServer);
  gulp.watch(baseSrc + '/**/*.js').on('change', gulp.series('build', browserSync.reload));
});

[phated]

@mmfilesi that is definitely a hacky and incorrect way to work around this. Just wrap browsersync.reload into an async function to do it the correct way.

[callumacrae]

@phated: The browsersync.reload function is designed to be used like that: https://www.browsersync.io/docs/gulp

That said, not sure if it's a good idea still.

[phated]

@callumacrae incorrect since we changed the way watch behaves (to bring it inline with task completion behavior).

[callumacrae]

Fair enough. @shakyShane are you able to update those docs?

[seantw]

There is no better way than require('gulp-watch') under Windows!

[guillegr123]

It seems to be a chokidar issue after all, when mixing windows-style paths and globs. See: https://github.com/paulmillr/chokidar/issues/777