Esbuild: [Feature] Code splitting on async import() statements.

[tracker1]

Support code splitting on dynamic import() statements, and additionally split/join on shared bundles for shared dependency models.

[evanw]

This is definitely something I plan to get to because I want to be able to use it myself. Right now import(path) turns into Promise.resolve().then(() => require(path)) so dynamic imports still "work" although they don't result in additional bundles. In the future it will generate separate bundles. I may also add support for common chunk and/or more advanced shared dependency analysis.

[jpmaga]

@evanw do you have any kind of roadmap somewhere for esbuild? I am particularly interested in this feature, and would be cool to know where it is in terms of planning. Cheers.

[andrewvarga]

This would be awesome to have!

[evanw]

I don't have a specific date but I'm currently focused on a rewrite of the bundler to enable code splitting, tree shaking, ES6 module export, and a few other features. I have to do these together because they are all interrelated.

I've done the R&D prototype to prove it out and I've settled on an approach. I'm currently working on doing the rewrite for real on a local branch. There's still a lot left to do to not break features I've added in the meantime (stdin/stdout support, transform API, etc) so it'll take a while. I have a lot of test failures to work through :)

I was worried about the performance hit because the graph analysis algorithms inherently reduce parallelism, but some early performance measurements seem to indicate that it won't slow it down that much, if any. I hope to ship this sometime in the next few weeks. We'll see how it goes!

[jpmaga]

I don't have a specific date but I'm currently focused on a rewrite of the bundler to enable code splitting, tree shaking, ES6 module export, and a few other features. I have to do these together because they are all interrelated.

I've done the R&D prototype to prove it out and I've settled on an approach. I'm currently working on doing the rewrite for real on a local branch. There's still a lot left to do to not break features I've added in the meantime (stdin/stdout support, transform API, etc) so it'll take a while. I have a lot of test failures to work through :)

I was worried about the performance hit because the graph analysis algorithms inherently reduce parallelism, but some early performance measurements seem to indicate that it won't slow it down that much, if any. I hope to ship this sometime in the next few weeks. We'll see how it goes!

Damn! You're the man. This is the only thing I am missing to start using it in production, in smaller projects for starters, and see how it goes. PS: Have tested a couple locally, without code splitting, and everything worked flawlessly, even in one with a fairly large codebase using react and typescript. 👍

[ponsifiax]

Hello here,
Do you have any news about this?
That the last feature to use it on production :+1:

[evanw]

Do you have any news about this?

It's mostly working already. The chunk splitting analysis has already landed. All that's left is to bind imports and exports across chunks. I'm working on that in a branch and this will be my main focus soon.

[evanw]

I just released version 0.5.15 with an experimental version of code splitting. See the release notes for details. It's still a work in progress but it's far enough along now that it's ready for feedback. Please try it out and let me know what you think.

[garygreen]

Excellent news! Thank you for all your hard work on this Evan. Code splitting was vital for us. Does this code splitting feature split css imports into seperate files and add at runtime? Simple CSS support is the next main thing we are eaglerly looking forward to.

[evanw]

Simple CSS support is the next main thing we are eaglerly looking forward to.

You and me both! CSS support is currently the next major feature I want to implement after code splitting. That’s tracked by a separate issue, however: #20.

[matthiasg]

Works really well in initial testing. We will test more complicated setups (rush repo, nested pnpm deps) more fully in the next weeks

[evanw]

That's great to hear! Thanks so much for trying it out.

[evanw]

I have a small progress update on code splitting. From the release notes for the upcoming release (not out yet):

Code that is shared between multiple entry points is separated out into "chunk" files when code splitting is enabled. These files are named chunk.HASH.js where HASH is a string of characters derived from a hash (e.g. chunk.iJkFSV6U.js).

Previously the hash was computed from the paths of all entry points which needed that chunk. This was done because it was a simple way to ensure that each chunk was unique, since each chunk represents shared code from a unique set of entry points. But it meant that changing the contents of the chunk did not cause the chunk name to change.

Now the hash is computed from the contents of the chunk file instead. This better aligns esbuild with the behavior of other bundlers. If changing the contents of the file always causes the name to change, you can serve these files with a very large max-age so the browser knows to never re-request them from your server if they are already cached.

Note that the names of entry points _do not_ currently contain a hash, so this optimization does not apply to entry points. Do not serve entry point files with a very large max-age or the browser may not re-request them even when they are updated. Including a hash in the names of entry point files has not been done in this release because that would be a breaking change. This release is an intermediate step to a state where all output file names contain content hashes.

The reason why this hasn't been done before now is because this change makes chunk generation more complex. Generating the contents of a chunk involves generating import statements for the other chunks which that chunk depends on. However, if chunk names now include a content hash, chunk generation must wait until the dependency chunks have finished. This more complex behavior has now been implemented.

Care was taken to still parallelize as much as possible despite parts of the code having to block. Each input file in a chunk is still printed to a string fully in parallel. Waiting was only introduced in the chunk assembly stage where input file strings are joined together. In practice, this change doesn't appear to have slowed down esbuild by a noticeable amount.

[matthiasg]

@evanw Thanks a lot for this detailed write-up ! This is the kind of information required for using a tool such as this.

[evanw]

Another code splitting update:

I finally got around to implementing per-chunk symbol renaming, which I view as required for the code splitting feature. I've made several attempts at this in the past but I haven't landed them because I don't want to severely regress performance (or memory usage, which I've started to also pay attention to). I finally figured out a good algorithm for doing per-chunk symbol renaming that's fast and parallelizable while not using too much memory. It's actually two algorithms, one when minifying and a different one when not minifying.

From the release notes:

Previously, bundling with code splitting assigned minified names using a single frequency distribution calculated across all chunks. This meant that typical code changes in one chunk would often cause the contents of all chunks to change, which negated some of the benefits of the browser cache.

Now symbol renaming (both minified and not minified) is done separately per chunk. It was challenging to implement this without making esbuild a lot slower and causing it to use a lot more memory. Symbol renaming has been mostly rewritten to accomplish this and appears to actually usually use a little less memory and run a bit faster than before, even for code splitting builds that generate a lot of chunks. In addition, minified chunks are now slightly smaller because a given minified name can now be reused by multiple chunks.

[guybedford]

@evanw it would be very interesting if you could expand somewhere on the exact symbol naming technique you converged on here. I'm sure it will make sense looking at the outputs too though of course.

[evanw]

@evanw it would be very interesting if you could expand somewhere on the exact symbol naming technique you converged on here.

I just wrote up some documentation about the parallel symbol minification algorithm here.

The non-minified symbol renaming algorithm isn't described in the docs yet but it's pretty simple. Just rename symbols to avoid collisions by appending an increasing number to the name until there's no longer a collision. Each symbol will need to check for collisions in all parent scopes. Symbols in top-level scopes must be renamed in serial but symbols in nested scopes can be renamed in parallel.

[mattiasewers]

@evanw Do you plan on supporting code splitting with other formats apart from esm?

[evanw]

@evanw Do you plan on supporting code splitting with other formats apart from esm?

Yes, that's why this issue is still open. However I want to fix issues with the current esm code splitting first: #399.