Esbuild: [MVP] Watch mode

[evanw]

I want esbuild to demonstrate that it's possible for a normal web development workflow to have high-performance tools. I consider watch mode a part of my initial MVP feature set for esbuild since I believe development builds should be virtually instant. This issue tracks the watch mode part of my MVP.

Watch mode involves automatically rebuilding when files on disk have been changed. While esbuild is much faster than other bundlers, very large code bases may still take around a second to build. I consider that too long for a development workflow. Watch mode will keep the previous build in memory and only rebuild the changed files between builds.

This may or may not involve disabling certain cross-file optimizations while in watch mode depending on what the performance looks like. I may also implement a local HTTP server that can be queried for the latest build, which is a nice way to avoid a page reload accidentally picking up a stale build.

[zmitry]

What about persistent cache and tool like this https://github.com/cortesi/modd ?

[progrium]

I was going to pull esbuild into my hotweb project which is basically this. Not sure how you'd like to separate concerns or if you want to just borrow each others code or what: https://github.com/progrium/hotweb

[gerardo-junior]

I do to do this very simply with browsersync with live reload

bs.watch('src/**/*.js', function (event, file) {
    require('esbuild').build({
         stdio: 'inherit',
         entryPoints: ['./src/scripts/index.js'],
         outfile: `${dist}/assets/scripts${!('development' === process.env.NODE_ENV) ? '.min' : ''}.js`,
         minify: !('development' === process.env.NODE_ENV),
         bundle: true,
         sourcemap: 'development' === process.env.NODE_ENV
    }).then(() => bs.reload())
      .catch(() => process.exit(1))
})

[zandaqo]

Watch mode involves automatically rebuilding when files on disk have been changed. While esbuild is much faster than other bundlers, very large code bases may still take around a second to build. I consider that too long for a development workflow. Watch mode will keep the previous build in memory and only rebuild the changed files between builds.

I understand the motivation and support it in principle, but I'd question prioritizing the watch mode over, say, tree shaking, aliases, or other vital features for a modern bundler.

With ~100x performance boost over existing bundlers, using esbuild without watch mode is already faster (e.g. 50 ms for me) than other bundlers operating in watch mode (750ms for me with Webpack). As it stands, though, without tree shaking esbuild produces bundles multiple times bigger than webpack, hence, I can only use it in development. But if speeding up my dev builds was my only goal, I would have gone with Snowpack... and it would make my already complex webpack setup a nightmare come update time.

That's where I think esbuild can help--reduce the complexity of our Snowpack+Webpack+Babel setups if it manages to give reasonable dev build speeds and feature coverage. Personally, I target modern browsers, use ES2019 and native web components with lit-element, thus, I require no compilation and ask my bundler only to, well, bundle all my code and minimize it at least by removing dead code I'm getting from third-party libraries.

[evanw]

I understand the motivation and support it in principle, but I'd question prioritizing the watch mode over, say, tree shaking, aliases, or other vital features for a modern bundler.

I’m totally with you. I’m actually already actively working on tree shaking, code splitting, and es6 export (they kind of have to all be done together since they are all related). Don’t worry!

[obnoxiousnerd]

A workaround for watch mode:
https://dev.to/obnoxiousnerd/watch-and-build-code-with-esbuild-2h6k

[tonyhb]

I'd rely on watchexec here - it's a _really_ good utility to listen to file changes built with rust. Very lightweight, very fast, very simple.

[koenbok]

We have a fairly large code base that takes 1-2s to build, which is already amazing. But we're definitely interested to bring this down even more with a great watch mode.

[zmitry]

Any updates on this? I want to integrate esbuild into my development workflow (use esbuild for development and storybooks) but without watch it requires some extra effort to get it working and with 2s build time it's almost the same as with webpack.
As for plugins, I found that I don't really need that feature as well as css output, I could handle that externally with my existing setup. The only deal breaker is watch mod at the moment.

[koenbok]

Our initial results here are cold build (~60s), warm build (~20s) and rebuild (~0.5s) all became ~2s. This alone is enough for us to make it worth the switch, even in watch mode, because it seems like the cold build savings (after a branch switch) make it a net win. That said, having a real watch mode would be amazing.

[rtsao]

I suppose the first step here is figuring out the plan for file watching. Unfortunately, it looks like fsnotnify, the most popular Golang file watcher (from what I can tell) doesn't yet support FSEvents on macOS. I think without this, a tool like esbuild is almost certainly going to run into issues with file descriptor limits.

Personally, I think it'd be fine to just use watchman, but I can see how imposing a hard external dependency on users might be unpalatable. FWIW, Parcel implemented their own C++ file watcher to avoid a hard dependency on watchman.

[evanw]

I've heard a lot about how flaky file watching is, which makes me hesitate to integrate file watching into esbuild itself. The primary mechanism that should be in esbuild core is incremental builds and a way to trigger them.

Personally I've found that it's more reliable to trigger incremental builds for development by running a local server that does a build when a HTTP request comes in instead of using a file watcher. Then the file is never out of date due to a race between the build tool and the loading of the file. That's what I'm hoping to build first as far as watch mode. Perhaps something like esbuild --serve=localhost:8000? It should be easy to do watch mode correctly and reliably using a local server without the issues people normally encounter with file system watchers.

The hard part is doing incremental builds. Once that is in place, it should be relatively straightforward to allow external code to trigger an incremental build (e.g. with a signal) and then use that to hook an existing external file watching library to esbuild without having to build the file watcher into esbuild itself.

[haggholm]

Maybe a balance could be struck between “an actual project build MVP needs file watching” and “esbuild should not contain all the logic for file watching” by maintaining, say, a lightly curated set of recipes for things like “esbuild+watchman”, “esbuild+chokidar”, and other (not necessary watch related) things that should not be part of esbuild but are the kinds of questions that everyone using it is likely to ask?

[rtsao]

The simplest possible form of incremental builds I can think of would be to merely skip re-bundling chunks that should not be changed at all. At least in cases where the dependency graph itself is unchanged, it should be straightforward to identify which chunks should be ultimately unaffected.

This a rather coarse form of incremental builds, but in scenarios where the majority of the bundle is from node_modules, the amount of skipped work could be substantial (provided node_modules are in a separate chunk from the modified code). Given esbuild is already extremely fast, I wonder if it might be "good enough" in practice until something more granular could be implemented. I'm not too familiar with the bundler logic, but I wonder if this more limited form of incremental builds would be easier to implement without making too many changes.

[laurentpayot]

For now I'm watching my TS files with Chokidar CLI to trigger full esbuild builds at changes and it is so fast that it does the trick for me (I'm bundling Firebase with a web worker and a few files).