Node: fs.watch only fires once on CentOS

[whtiehack]

• Version:v8.10.0 and v10.9.0
• Platform:Linux HDServer 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
CentOS release 6.8 (Final)
• Subsystem:

test code:

const file = '/home/qianqiu/testnodejs/testwatch/test.json';

const fs = require('fs')

let watcher = null;
const fun = (event,name)=>{
if(event!='change') return;
console.log('!!! file event:',event,name);
};
watcher = fs.watch(file,fun);

I modify the file and only watch for the event once. There are no events to modify the file twice.

---

My solution:

const file = '/home/qianqiu/testnodejs/testwatch/test.json';

const fs = require('fs')

let watcher = null;
const fun = (event,name)=>{
if(event!='change') return;
console.log('!!! file event:',event,name);
watcher.close();
watcher = fs.watch(file,fun);
};
watcher = fs.watch(file,fun);

Is this a bug?
thanks.

[ghost]

Tested on Windows 7, 8, 8.1, 10, Ubuntu 16.0, 17.0 using your code with manual file modificating and automated file modificating and unable to reproduce the issue. How do you exactly modify the file?

[whtiehack]

vi test.json
do something , and : wq

Windows7 has no problems. It is now the same problem tested on centos 6.3.
CentOS release 6.3 (Final)
Linux VM_216_17_centos 2.6.32-431.23.3.el6.centos.plus.x86_64 #1 SMP Wed Jul 30 00:12:13 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

v8.11.1

[whtiehack]

It is now the same problem tested on CentOS Linux release 7.4.1708 (Core).
Linux iZbp1b0a77gz72g12vqn8kZ 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
v6.14.3

[jiripospisil]

Does that happen with just Vi or other editors as well? Does setting nowritebackup in Vi make any difference?

set nowritebackup

[whtiehack]

@jiripospisil set nowritebackup Setting nowritebackup is normal.
But usually if you change the file with git pull. watch can only be listened once.
So what is this problem?
Thank you for the answer.

[davisjam]

Summary

I cannot reproduce this on Ubuntu 16.04.5 running v8.9.3 or v10.9.0.

Recreate efforts

I used the script provided.

$ cat watch.js 
const file = '/tmp/file-to-watch';

const fs = require('fs')

let watcher = null;
const fun = (event, name) => {
  if (event != 'change') return;
  console.log('!!! file event:', event, name);
};

watcher = fs.watch(file,fun);

I ran touch /tmp/file-to-watch, echo a >> /tmp/file-to-watch, and a simple vim-based interaction. All of these interactions resulted in change notifications from the watcher.

@whtiehack Any chance you have an Ubuntu machine available? It's vaguely conceivable that this is some distro variation in inotify semantics.

[whtiehack]

@davisjam Now I do not have other linux machine.

I tested it on the docker VM on Windows and the test results were correct.
Linux default 4.9.93-boot2docker #1 SMP Thu May 10 16:27:54 UTC 2018 x86_64 GNU/Linux
Boot2Docker 18.05.0-ce (TCL 8.2.1); HEAD : b5d6989 - Thu May 10 16:35:28 UTC 2018

So, this should be a problem with the Centos only?

[davisjam]

@whtiehack Here is an automated test so you can try to reproduce on your machine.

1. Create files /tmp/watch.js and /tmp/run.sh with the following content.
2. Make them both executable.
3. Run '/tmp/run.sh' and paste your output here.

$ cat watch.js 
#!/usr/bin/env node

const fs = require('fs')

const file = process.argv[2];
console.log(`Watching ${file}`);

let watcher = null;
const fun = (event, name) => {
  if (event != 'change') return;
  console.log('!!! file event:', event, name);
};

watcher = fs.watch(file,fun);

$ cat run.sh 
#!/usr/bin/env bash

set -e

WATCH_FILE=/tmp/file-to-watch
LOG_FILE=/tmp/watch.log

touch $WATCH_FILE
node watch.js $WATCH_FILE > $LOG_FILE 2>&1 &
sleep 1;

touch $WATCH_FILE
sleep 1
echo a >> $WATCH_FILE
sleep 1
echo b >> $WATCH_FILE
sleep 1

cat $LOG_FILE
echo "Found this many file events:"
grep -c 'file event' $LOG_FILE

set +e
ps -ef | grep watch.js | awk '{print $2}' | xargs kill -9
exit 0

The 'grep' command should yield "3" to the console.

This is what happens on my Ubuntu machine:

$ ./run.sh 
Watching /tmp/file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
Found this many file events:
3

[davisjam]

@whtiehack

I installed a CentOS VM (CentOS 7, kernel 3.10.0-862.el7.x86_64) and ran these scripts. on Node v10.9.0 and Node v8.11.4. The fs.watch behavior is as expected -- 3 events, the same as on my Ubuntu box.

I cannot reproduce the problem you report.
Let me know what you find when you run my run.sh script.

[whtiehack]

CentOS release 6.8 (Final)

[root@HDServer tmp]# ./run.sh 
Watching /tmp/file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
Found this many file events:
3
kill 16336: 没有那个进程

CentOS Linux release 7.4.1708 (Core)

[root@iZbp1b0a77gz72g12vqn8kZ tmp]# ./run.sh 
Watching /tmp/file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
Found this many file events:
3
kill: sending signal to 13861 failed: No such process
./run.sh: line 24: 13847 Killed                  node watch.js $WATCH_FILE > $LOG_FILE 2>&1

CentOS release 6.3 (Final)

[pinus@VM_216_17_centos tmp]$ ./run.sh 
Watching /tmp/file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
!!! file event: change file-to-watch
Found this many file events:
3
kill 27868: 没有那个进程

The fs.watch behavior is as expected(3 events).

I think the problem is vi in CentOS, when i use setting nowritebackup in vi, the behavior is as expected.

So this may not be a bug, just the weird behavior of vi on CentOS?
Maybe fs.wach needs to 'delete' or 'remove' events instead of rename twice?
Thank you @davisjam .

[jiripospisil]

@whtiehack Setting nowritebackup makes it such that Vi writes directly to the opened file when saving. If not set, it first writes to a temporary file and then replaces the opened file with the temporary one. This probably interferes with the watch mechanism because it does't care about file names but rather the underlying inodes of the files. In other words you're watching inode of file A, it's then replaced with file B with the new content, and at this point B is the "current" file but you're still watching inode of A.

Try removing the line where you filter for the change event to see whether there's some replace or move event or something along those lines.

[whtiehack]

@jiripospisil Thanks for the answer, it should be what you said. When I remove the filter,there are three events, change,rename and rename,same as use git pull. So, I think needs to delete or remove events instead of rename twice.

[davisjam]

@whtiehack As I understand it, there is no Node.js issue here. OK to close this?

[whtiehack]

@davisjam Thanks.

[vin-d]

Just got blocked by this issue too. I confirm this (https://github.com/nodejs/node/issues/22517#issuecomment-416480323).
I'm on Ubuntu 18.04 and I am using WebStorm 2019.1 to edit the file and do my tests.
I could only get notified of the first edit to my file so I thought the documentation was broken as it was clearly not persistent watch.

with the command stat my file.txt
I see that Webstorm changes the Inode number when it saves my changes.
That's a crappy problem.
Seeing that I've retried by editing with nano, and that goes fine as nano keeps the same Inode.

It would be great if Node would have both tracking features : fs.watchByInode and fs.watchByPath.
This could be done easily because when the "replace-edit" is done by WebStorm and vi there is a last event fired on the original inode. Then Node could check on the filepath to ensure that the same inode number is still used, and update it's watching routine if it's a new inode.

Both watching routines could be useful depending on the development/OS requirements, also this would be explicitly documented.

[whtiehack]

A simple solution is to watch the root directory and then judge by file name.

Such as if you want watch /data/file.txt,then you can watch /data ,fs.watch('/data',(event,name)=>{if(name=='file.txt'){}}.