Emscripten: Support ES6 in JS optimization (Babel? Acorn?)

[saschanaz]

With Babel, we can:

• use ES2015+ freely and replace Module['print']('fetch-worker sent a message: ' + e.filename + ':' + e.lineno + ': ' + e.message); into Module['print'](`fetch-worker sent a message: ${e.filename}:${e.lineno}: ${e.message}`);, etc.
• use JS-syntax-compatible conditional compilation (#5732)

[curiousdannii]

Can also replace a lot of custom minification (including DCE) with babel-minify.

[kripken]

We would need to migrate the few JS optimizer passes that run on general JS, which includes JSDCE. But that shouldn't be too hard. (The great majority of JS optimizer passes only run on asm.js, so they aren't blocking us from using ES6 etc.)

How good is babel-minify? I assume we'd still need closure compiler in this case? How do those integrate?

Also, is babel the best tool in this space these days? A few years ago there were a bunch of JS ASTs and optimizers, I haven't followed this space closely for a while.

[curiousdannii]

Babel has almost all the market share in transpiling/polyfilling. It shouldn't be too hard to add an emcc option to take a list of browsers, pass that to Babel, and then have it automatically transform any unsupported syntax.

Babel-minify is pretty good, but it's not that stable. But it is implemented just as a bunch of babel plugins, so that problematic ones can be easily disabled. It doesn't support asm.js. Probably it could be supported, but I haven't determined yet which plugin(s) are the problematic ones.

uglify is used way more, but IMO has toxic development practices.

If you want to output ES6, then (I think) Closure Compiler isn't an option. If you like saschanaz's idea about conditional compilation, then Babel would probably be an excellent choice.

[rongjiecomputer]

No objection with using Babel for conditional compilation, but please make sure it is possible to disable transpiling/polyfilling/minifying. I hope Babel does not strip comments as I need some type annotations in comments for Closure Compiler.

Closure Compiler can already output ES6 code, though less optimizations are available in ES6 mode and async function is not yet supported. Commit history shows that Closure Compiler team is actively working on ES6 mode to enable more optimizations. Not sure if they are working on async function.

[saschanaz]

Closure Compiler can already output ES6 code

Its wiki says no:

You cannot use it as an output language yet; for now you need to transpile down to ES5 or ES3.

But anyway I think the purpose here is to input ES2015+, not output.

[rongjiecomputer]

@saschanaz The wiki page was not updated since May.

If you go to the following link, you can see that Closure Compiler can understand and optimize ES6 without transpiling. For java binary, pass --language_in=ECMASCRIPT6 --language_out=ECMASCRIPT6.

https://closure-compiler.appspot.com/home#code%3D%252F%252F%2520%253D%253DClosureCompiler%253D%253D%250A%252F%252F%2520%2540compilation_level%2520ADVANCED_OPTIMIZATIONS%250A%252F%252F%2520%2540warning_level%2520VERBOSE%250A%252F%252F%2520%2540language%2520ECMASCRIPT6%250A%252F%252F%2520%2540language_out%2520ECMASCRIPT6%250A%252F%252F%2520%2540use_types_for_optimization%2520true%250A%252F%252F%2520%253D%253D%252FClosureCompiler%253D%253D%250A%250Aclass%2520Hello%2520%257B%250A%2520%2520constructor(x%2520%253D%252010)%2520%257B%250A%2520%2520%2520%2520this.x%2520%253D%2520x%253B%250A%2520%2520%257D%250A%2520%2520sayHello()%2520%257B%250A%2520%2520%2520%2520console.log(%2522Hello%2520%2522%252C%2520this.x)%253B%250A%2520%2520%257D%250A%257D%253B%250A%250Aconst%2520h%2520%253D%2520new%2520Hello(%2522Jarvis%2522)%253B%250Ah.sayHello()%253B

[kripken]

We should remember to look into source map support when investigating Babel (see #4224, currently our source maps break when the JS optimizer is run - hopefully with Babel there's a good way to avoid that).

[kripken]

Talking to @yurydelendik today, we had some thoughts on this topic:

• As a high-level goal, we want emscripten to both emit something that fits well into web bundler build systems - for the library use case - but also can emit a fully optimized output by itself - for the application use case.
• It would be nice to use DCE from Babel etc., so we don't do it ourselves, but that is somewhat at odds with the application use case of emitting a fully optimized output by ourselves. On the other hand, that's why we support closure advanced (which is especially good for the application use case, where nothing else is being built but what closure sees when called by emcc). But still, we need some amount of JS opt passes for our metadce and other specific optimizations. That's pretty minimal, though, and we probably just want to keep doing what we are, and not invest in new big things there.
• To replace our current JS optimization which uses Uglify1, another option is Acorn (used by webpack and others). It provides an AST and is fast, and supports ES6 - so it should be possible to migrate the JS opt passes we need to there. It's also probably easy to include in emscripten (whereas I think Babel is larger and has more dependencies, but I'm not sure).

[curiousdannii]

I know hardly anything about this all, but it looks like many JS projects use related but slightly different parsers. Babel for example uses its own parser, but it is based on Acorn, and it outputs its own AST format, which in turn is based on ESTree (which Acorn uses).

I don't know how much these all overlap, but I'd expect a large extent. There's probably only a few cases where the differences are relevant to Emscripten, and as most of these tools do let you change parser, I'd expect they also expose the parser to each traversal/transformation function so the differences could be handled.

If the Emscripten passes could be ported to use the Acorn/ESTree APIs, then that should give lots of flexibility for the future.

One step in preparation would be for someone to do an audit of all the current passes to write up a list and give a brief explanation. Then we could more easily see what was essential vs optional, what needs to be handled in Emscripten (METADCE) vs what could be handed off to an independent tool, etc.

[kripken]

PR up: https://github.com/emscripten-core/emscripten/pull/7973