Emscripten: [Feature Request] ES6 support for the uglifier

[ThorstenBux]

To me it appears that the uglifier-js used in version 1.37.22 does not support ES6 syntax. It refuses to work with arrow-functions (=>), it refuses to work with default values for function parameters ( var x = function(x=1) ), it refuses to work with the new variable declarations ( let ), it refuses to work with template strings (backticks '`' eg: var s = `test ${i}` )

[kripken]

Correct, this is a current limitation. Some discussion about one possible way to solve this is in #6000, but currently as far as I know no one is working on it. It would be great if someone could.

[ThorstenBux]

Is it possible to change the uglifier? using this https://github.com/mishoo/UglifyJS2/tree/harmony for example?

[kripken]

Sadly no, it's not that easy. We use an older version, Uglify1, and not Uglify2. There's a big API difference between 1 and 2, so migrating to Uglify2 would be as easy as migrating to Babel as suggested in #6000.

With that said, neither is extremely hard. Probably a few weeks of work. Most of the work is to rewrite a small number of passes we have from Uglify1 to the new API - it's a small number since we don't need to modify any asm.js passes, just the ones that run on generic JS, not many of those.

[curiousdannii]

I didn't realise that this was an issue not just for the optimiser, but also for parsing library files. :(

[kripken]

Only when they are parsed by the optimizer, though. If JS isn't being optimized (-O0, -O1, or if JS opts are disabled in higher opt levels), then it's just copied through. Unless I'm missing something?

[curiousdannii]

Oh, you're right, sorry. I'm building with -O3.

[saschanaz]

Personally I don't like what Emscripten does with npm packages, can we install from npm instead of copy-pasting whole packages into the repo?

[kripken]

Yeah, in general it would be good to use npm properly for upstream things. The Uglify1 code we use is an exception though since it's been modified and exists only here (so I'm not sure of the benefit of using npm for it).

[curiousdannii]

How is UglifyJS called? Pass a string in, get a string out? Do we use the UglifyJS parser for any other purposes? If it's used mostly as a closed box then replacing it will be more straightforward.

[kripken]

We use it for asm.js in some ways that are irrelevant to this topic. For the non-asm.js stuff that is relevant, we have 2 main uses:

• Just use it to minify whitespace. We do that in -O2 and above by default. This is a black box use and is easily replaced.
• A few JS optimization passes run on the non-asm.js code. In these cases, Ugilfy generates an AST, which we then optimize, then pass back to Uglify to get the JS. To replace these, those passes would need to be rewritten to use the a new AST (of whatever thing we switch to). There are few passes here (JSDCE, the metadce helper code) and they are fairly short, and we have tests written that will make verifying the rewrites easy. But this will take some work, probably more than a few days.

[curiousdannii]

Just use it to minify whitespace. We do that in -O2 and above by default. This is a black box use and is easily replaced.

Is this the code, and could this function therefore be replaced with one using Babel? If so it could be an easy place to start. However it does receive the ast, not a string...

https://github.com/kripken/emscripten/blob/5952e749af724ee8c1c3838e7ed8497c816d051b/tools/js-optimizer.js#L171-L178

[kripken]

Yes, that's it. The ast is generated near the bottom of the file, as we read the input and parse it.

Probably we'd need a new file for Babel. So there would still be the current js-optimizer (still used on asm.js), and the new one would provide the same external interface (but just support the small set of passes we port to Babel).

[mathiasbynens]

This can be solved by switching from uglify to terser.

[kripken]

@mathiasbynens

This can be solved by switching from uglify to terser.

Looking into this now, I may be missing something, but terser does not seem to have an AST compatible with Uglify1. Reading

• https://github.com/terser-js/terser/blob/master/lib/ast.js
• https://github.com/terser-js/terser/blob/master/lib/parse.js

The AST looks similar to the SpiderMonkey AST (but is apparently not identical). For example, in Uglify1 a conditional would be ['conditional', x, y, z], while terser has { TYPE: 'conditional', condition: x, consequent: y, alternative: z [..] }.

So switching to terser seems similar to switching to Babel/Acorn/etc. - a rewrite of our JS optimizer passes. (Which isn't the end of the world - for wasm we have very few relevant passes, as discussed above. But if we had a compatible AST we could have done it in a day...)

[saschanaz]

Maybe we can write a compatibility layer for uglify1/uglify2(terser) to ease migration.

[kripken]

@saschanaz To convert between the ASTs? I think the hard part would be translating ES6-specific constructs there.

In any case, the crucial passes are probably just a few hundred lines of code - around a week or so of work, I'd guess, for someone familiar with both ASTs.

[curiousdannii]

Long term I think moving Emscripten to operating on Acorn ASTs would be best as then Babel would be an option too. Terser can accept an Acorn AST, but it does convert it internally. But as long as the Emscripten passes were run before Terser, that should still work I think?

[kripken]

PR up for this, feedback very welcome,

https://github.com/emscripten-core/emscripten/pull/7973

This issue is a duplicate of #6000 I believe, closing in favor of that one.