Bloop: Add support for file watching `~`

[jvican]

We should probably add a really fast source file watching for ~compile and ~test. We may want to add this directly to Zinc rather than experiment with it directly in blossom. I prefer this approach since it's more generic.

[olafurpg]

I would go with something native for every OS like https://facebook.github.io/watchman/

[jvican]

It has just occurred to me that source file watching should be supported by our tool with a cross-platform solution, while still allowing users to run a native solution that executes a compile command with bloop.

[jvican]

I've been looking into watchman and use it locally. I've liked a lot how it works. The fact that it works on all OSes makes me lean on it for now, for our first prototype.

[jvican]

I think support for file watching inside bloop should come in Bloop 0.2. For now, we should just have high-quality docs on how to use watchman with bloop + nailgun. What do you think @Duhemm? If you agree, I propose we move this to milestone 0.2.

[olafurpg]

I agree this can be postponed , esp. since it's possible to use watchman/entr with bloop + nailgun. I'm also eager to explore integrating lsp with bloop to run compile/test on file save. In that case the editor provides the event hook.

[Duhemm]

That sounds very reasonable 👍

[Jasper-M]

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

[jvican]

We're using directory-watcher because it claims speedups in OSX and does not do polling. I have a prototype with it I'll PR any time soon.

[fsvehla]

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

It uses polling under macOS and Linux, which leads to for example SBT using 40% of CPU constantly watching a medium size code base.

[jvican]

Does it under Linux? I think under linux it uses fsevents.

[fsvehla]

My information re: Linux might be outdated, I am not using it for development.

[gmethvin]

The OS differences are basically:

• Windows JDK WatchService does async watching and recursive watching (ExtendedWatchEventModifier.FILE_TREE).
• Linux JDK WatchService does async watching but not recursive watching. In practice you can handle the recursive case you can register a new WatchKey for directories when they are created.
• macOS JDK WatchService uses polling and at best can be configured to poll every two seconds. To get a decent experience you need to use a separate native program or JNI/JNA. directory-watcher provides/uses a JNA implementation that also supports recursive watching.

I think it makes sense to use a higher-level API, perhaps something like File.Monitor from better-files. (directory-watcher already implements that: https://github.com/gmethvin/directory-watcher/#better-files-integration-scala). Besides the fact that the WatchService API is not very nice, it's also useful to customize the recursive aspect so it uses the most efficient mechanism for each OS.

[jvican]

125 successfully implements this feature thanks to your work, great library 👍 @gmethvin. In the next days we'll experiment with file watching in bloop in larger projects and we'll see how it fares in comparison with the default file watching algorithms in sbt/sbt.