Chokidar: Add a file "rename" event

[jeremydiviney]

would be very conveinent if chokidar emitted 'rename' events when a file is renamed. Currently you get an "add" and "unlink" event, and it is challenging to detect that a file was renamed. Especially because it seems that the "unlink" and "add" events don't always emit in the same order when renaming a file.

[es128]

Can't get this consistently from the underlying APIs (primarily from node core), and replacing the current add/unlink with a rename would be a breaking change. Keeping the behavior as consistent as we can regardless of platform is a high priority, so even in the circumstances where this is possible it's unlikely to get exposed by chokidar.

[es128]

Open to ideas though, and maybe there is a strategy that would allow this to be added as a non-breaking change (such as being enabled by an option or emitting rename in addition to the other two existing events).

[CxRes]

Since I desperately could do with a rename event as well, may I suggest at the very least you can add a rename argument to add and unlink events as a boolean. Then, if you add a rename event, the user may just choose to add a empty return if rename flag is true in the onAdd and onUnlink routines and let it be handled by the rename event's onRename callback.
For those who don't care, the api does not break as add and delete are still emitted! Also gives the user flexibility to let common code between add/unlink and rename to remain in the former.

[paulmillr]

I think it is possible now to make the event on your own. Try something like this:

var recentlyMoved = {};
var recentlyMovedTimers = {};

var removeRecentlyMoved = function(path) {
  if (!recentlyMovedTimers[path]) return;
  clearTimeout(recentlyMovedTimers[path]);
  delete recentlyMovedTimers[path]
  delete removeRecentlyMoved[path];
};

var checkIfWasMoved = function(path) {
  if (recentlyMoved[path]) {
    emit('rename', path);
    removeRecentlyMoved(path);
  } else {
    recentlyMoved[path] = true;
    recentlyMovedTimers[path] = setTimeout(removeRecentlyMoved.bind(null, path), 1000);
  }
};

watcher.on('add', checkIfWasMoved);
watcher.on('remove', checkIfWasMoved);

[es128]

@CxRes that's a really good idea for how to handle the API when the data is available. I remain concerned that it will be difficult to relate the corresponding rename, add and unlink events deterministically. But this is now on my to-do list of stuff to try out. And PR welcome if anyone else can figure it out sooner.

@paulmillr it's a nice idea, but also non-deterministic I'm afraid. Looks very vulnerable to races and edge cases.

[CxRes]

@es128 I too am having second thoughts about having code common between add & rename or unlink & rename. Well, a little code repetition is a small price to pay for this!!!

But, I think there is no problem, if the user is prudent enough to do the following as long as Chokidar can guarantee that oldfile is becoming newfile . I do not see see a race condition since only one block will execute. However, knowing if this condition (oldfile -> newfile) can be met is your area of expertise:

watcher.on('add', function (newfile,stat,rename){
  if (rename) return;
  else{...}
});

watcher.on('unlink', function (oldfile,rename){
  if (rename) return;
  else{...}
});

watcher.on('rename', function (oldfile,newfile,stat){
 //manage your rename event
});

On an unrelated note, I find that having addDir/UnlinkDir somehow unstatisfactory - given that on this occasion you will have to write renameDir as well. I end up using same call back for Dir and Files on many occasions and certainly see that happen for rename. I wonder why that option was taken instead of using the stat object to detect directories (imho, though I may be wrong saying this: even unlink/change should stat the historical object!)

[es128]

Code repitition was not my concern - it's controlling the flow of events such that chokidar can guarantee that the add and unlink related to a rename wind up being flagged as such, as each will be detected along a separate flows of logic. I'm talking about how it would be within chokidar's code, not the user-facing perspective.

It's not like chokidar is taking underlying rename events now and translating that to the two separate events - if that were the case this would probably be easier. I don't know since I haven't really examined the situation yet, but it may be hard to do this without resorting to artificial delays and other such undesirable artifacts of supporting this feature. I do intend to spend some time finding out sooner or later though.

[CxRes]

Oh ok! I am afraid that even after staring at the code I do not understand the internals. Thanks for looking into it and would really appreciate a rename event soon.

[es128]

You could try @paulmillr's sample code to begin with and see if it works out for your particular use-case.

I wouldn't expect this to be resolved very soon within chokidar (not by me, anyway).

[Xample]

same needs here, or if only the events were always arriving in the same order…

[nono]

I'm also interested by a "rename" event.

@es128 @paulmillr the sample code does not work. If I understand correctly, it detects files that are created and removed in the same second (and deleted and recreated).

[paulmillr]

@nono the problem is: there is no "file system level" rename event per se, for all platforms and cases. So, this may be the best thing we can create

[nono]

Well, it's a bit more complicated than that:

• With inotify, there are IN_MOVED_FROM and IN_MOVED_TO attribute,
• The fsevents lib on npm has moved-in and moved-out events,
• For windows, I don't know,
• When using polling, it's probably very difficult to detect move/rename in a reliable way.

I understand that it needs a lot of code and effort to have something usable and that masks the differences between the platform. Maybe using timers is a good way for handling renames. But I still think it should be included in chokidar, as it's not easy to get it right.

It's like awaitWriteFinish: it's code that can live out of the chokidar repository. But the first version was buggy and contributions from the community helped to fix the bugs.

[es128]

@nono I think it would be ok to explore ways to optionally expose the event in the modes/platforms where it is already made available by the lower-level layers, but then we have to take great care to reconcile the inconsistencies in behavior that could lead to.

I do not think it will be possible to successfully shim this for situations like polling that do not provide the information.

So a rename event may be useful for situations where the environment is controlled/known and supportive of reporting renames, but if building cross-compatible applications, then it will continue to be necessary to account for cases where renames are reported as unlink and add.

[zhirzh]

What about using inode numbers? We can compare them to check whether the paths of subsequent add and unlink events point to the same file, i.e., have same inode number.

I know and I have checked that inode numbers stay the same for the following processes on both, files and directories under *nix environment:

• cp - copy
• mv - move
• rename - rename
• ln -s - symbolic link

Files only:

• ln - hard links

I am certain that windows doesn't have inode numbers and I strongly believe that windows lacks an alternative to it (please prove me wrong, internet).

[popod]

Any update on this issue ? I've the same request: I would like to know if a files is renamed/moved or added/removed..

For infos: it seems that fs.watch has a "rename" event.

[paulmillr]

WONTFIX. Unreliable. Feel free to implement it on the user side.

[NeekSandhu]

I wonder why no one paid attention to @zhirzh comment:

What about using inode numbers? We can compare them to check whether the paths of subsequent add and unlink events point to the same file, i.e., have same inode number.

I tested stating a file before and after rename on Windows, macOS and Ubuntu. inode numbers didn't change!

Seems absolutely legit, what is stopping us?

[paulmillr]

Feel free to fork chokidar and send a pull request. If it gets used and tested reliably, that could be fine.

[nono]

@NeekSandhu you should also test that inode numbers are not reused. IIRC, you can safely use them on Linux, but Windows reuse them (ie, if a file is deleted and another file is added just after that, they can have the same inode number even if they are not the same file).

[CxRes]

@nono @NeekSandhu How about using inode and ctime together? If inode gets reused surely the ctime changes unless it is a rename? At least one might reasonably expect that...

[NeekSandhu]

@nono Great discovery.
@CxRes Did you mean birthtime instead?

But how do we make sure we don't break the API. Since having this will no longer emit unlink events (by design) when any entity is renamed.

[CxRes]

@NeekSandhu You are right! I did not know that fsStat gave us birthtime as well.

regarding API: I had made one suggestion above which @es128 agrees with. Alternatively one can have a boolean flag detectRenames like @NeekSandhu suggests (in the unedited comment).

[arxpoetica]

My two cents. The API should absolutely be broken. Chock it up (no pun intended) to what a library is supposed to do: upgrade semver when the need is real. This is a very appropriate, and I would argue, correct example to break the API over.

[glortho]

My 2¢: After doing lots of research I can say that cross-platform rename in node leaves much to be desired. We couldn't find a way to do it without timeouts, which creates dangerous race conditions.

FWIW we went with https://github.com/gorakhargosh/watchdog, which emits a moved event along with old and new path (using kqueue/fsevents on Mac, inotify on Linux, etc. along the lines @nono's comment https://github.com/paulmillr/chokidar/issues/303#issuecomment-164497736)

[fabiospampinato]

I've implemented this here, it was somewhat tricky as the order of the "add" and "unlink" event is not consistent across platforms. I'm probably going to publish it on NPM for easier consumption in the future.

[nono]

For your information, several files/folders can have the same ino on windows because of rounding issues of large numbers in JS. More details on https://github.com/nodejs/node/issues/12115

[fabiospampinato]

@nono thanks for pointing that out! It seems the problem would be solvable if the stats object were to include bigints, any way to make sure chokidar provides me stats objects with bigints?

[nono]

You can use make a pull request for making chokidar calls fs.stat with the bigint: true option. And be sure to use a recent version of nodejs, this option was introduced in node v10.5.0.