Video-hub-app: Current chokidar parameters need to be tweaked for performance

[cal2195]

I've been experimenting with the new VHA today, and due to the current chokidar config, it's been crashing my systems kernels with an overload of filesystem operations! 😅

I'm working on a better set of options, but here's some of the issues so far:

• The ignored list has changed, so now it only ignores files (vha-*, images, etc.) after already adding a inotify watcher on it, and doesn't release the watch. This is a massive performance drop as it indexes my multiple vha-* dirs for no reason.

• Some of the params, like alwaysStat and awaitWrite are causing too much activity at one time, and would be better suited dealing with later in the process.

I'll update this with more findings as I experiment! 👍

[cal2195]

Could be related to #512

[cal2195]

After tweaking a few settings, I've gone from consistent system-wide crashing, to a full scan in a few minutes! 🎉

[whyboris]

@cal2195 -- could you share the settings that prevented the system-wide crashing 😅 🙇 ?

[cal2195]

Sorry, this slipped past my inbox! 😄

Looking at my repo, the main params I changed in watcherConfig were:

• alwaysStat: False - not 100% sure on this one, but I think we re-stat ourselves if it's not present anyway. I think this was causing too many stats in a short space of time.
• awaitWriteFinish: False - not worth the potential benefit, causes way too much of a performance hit.
• ignored: /vha-|\.jpg|\.tags/ - Very important - ignore these files before processing them, otherwise you get well over 4 times the files to process that in your library, even though the first thing the processor does is skip them.

Also, moving the console.log(fullPath) in the watcher.on('add') method up to the top means we can track all paths being processed before they're just dropped to improve that ignored param! 👍

[whyboris]

Thank you @cal2195 again for excellent advice 🙇

I've addressed it with #541 -- if you have a moment, please glance to see if it looks ok 🤝 😁