Uglifyjs: Hangs with 100% cpu usage on webpack js bundle with compress: true
Bug report or feature request?
Bug report
ES5 or ES6+ input?
ES6, but it doesn't matter. ES5 has the same problem.
Uglify version (uglifyjs -V)
3.2.2
JavaScript input
There are various reports in the webpack repository of uglify related hangs, see: https://github.com/webpack/webpack/issues/4558#issuecomment-352170623
https://github.com/webpack/webpack/issues/2012
Also here:
https://github.com/gdi2290/angular-starter/issues/1925
I've extracted the generated bundle from webpack prior to the uglifyication step and can reproduce this problem with uglifyjs2 alone, just by feeding it that input and passing { compress: true }.
This is a 12MB js file, which I can share to a developer that wants to take a look at privately (I'd rather not make my whole code entirely public). Let me know and I'll put it on dropbox or something.
I believe it is related to something in @angular/core somehow, because in the past while debugging this issue, I removed that dependency from my code and the uglification step passed without it.
EDIT: repro here
The uglifyjs CLI command executed or minify() options used.
{ compress: true }
JavaScript output or error produced.
Hangs with 100% cpu usage and memory usage balloons to 10GB+
andreialecu
All 16 comments
I've been playing with this more and found out that it's the unused compression option that causes this.
compress: {
sequences: true,
properties: true,
dead_code: true,
conditionals: true,
evaluate: true,
booleans: true,
loops: true,
unused: false, // setting this to true hangs uglify
hoist_funs: true,
if_return: true,
join_vars: true,
cascade: true,
warnings: true,
negate_iife: true
}
In order for me to look at this, please narrow it down so you can provide a publicly available test case.
Of course you are welcomed to file a Pull Request to fix this issue yourself.
alexlamsl
on 17 Dec 2017
Hangs with 100% cpu usage
...
This is a 12MB js file
I assume by "hangs" you mean it takes a long time? How long does it take to complete exactly?
kzc
on 17 Dec 2017
@kzc I haven't tried waiting more than around 10 minutes. Memory usage goes to 10gb in under 1 minute on the node process and then other processes usually end with out of memory exceptions, so most times things start crashing before getting a chance to see it end. It does GC after a while, so from 10GB it goes back to 1GB, then starts growing again.
With unused: false in the options, it finishes uglifying in around 10-15 seconds and memory usage is stable.
I'm trying to trim down my bundle and see if I can make a smaller repro.
andreialecu
on 17 Dec 2017
Recent uglify releases perform a lot more optimization than past versions. The compress options reduce_vars and collapse_vars in particular. Use whatever options best suit your project.
It's not uncommon to disable compress to speed up uglify.
https://github.com/mishoo/UglifyJS2#uglify-fast-minify-mode
kzc
on 17 Dec 2017
@kzc please read my second comment, it's being caused by unused: true in particular.
andreialecu
on 17 Dec 2017
This issue is not actionable without a publicly available test input file.
kzc
on 17 Dec 2017
@kzc @alexlamsl
Here's a repro:
https://gist.githubusercontent.com/andreialecu/3d8f4271eaf981089675bc7fcad06b66/raw/a57ffb5ea4ad326b092ded6f85815dbcf7f98779/test.js
Minify with these options:
{ ecma: 6, compress: {
sequences: true,
properties: true,
dead_code: true,
conditionals: true,
evaluate: true,
booleans: true,
loops: true,
unused: true, // false works
hoist_funs: true,
if_return: true,
join_vars: true,
cascade: true,
warnings: true,
negate_iife: true
} }
Converted the handful of ES6 syntax in the test file above, I can confirm uglify-js does not suffer any performance issue with this particular input:
$ uglifyjs test.js -mco min.js --timings
- parse: 0.562s
- rename: 0.453s
- compress: 4.719s
- scope: 0.266s
- mangle: 0.484s
- properties: 0.000s
- output: 0.484s
- total: 6.968s
uglify-es certainly seems to be stuck in a loop, though there is no monotonic increase in heap usage.
alexlamsl
on 17 Dec 2017
uglify-es + compress: true have been the default in webpack's uglify plugin for a few versions now. Angular uses webpack, so this is a pretty big issue for several communities.
Regarding the memory leak, I was mistaken, there seems to be none when running webpack normally. My apologies.
I've been debugging this for several days and seen the memory issue while running node --inspect-brk ..../webpack.js but I assume that is to be expected:
andreialecu
on 17 Dec 2017
$ cat repro.js
function REPRO(a) {
for (var b = [ u(Rh(a)), u(" ") ].join(""), c = n(Z(/^GN|KN|PN|WR|PS$/, Y(b, 0, 2))) ? 1 : 0, d = H(a), e = d - 1, g = c, h = Fc, k = Fc; ;) {
if (g > d || 4 <= H(h) && 4 <= H(k)) {
return new Q(null, 2, 5, R, [ Y(Id(u, P.a(Lf, h)), 0, 4), Y(Id(u, P.a(Lf, k)), 0, 4) ], null);
}
var m, q = b, p = g, z = d, B = e, A = Y(q, p, 1), G;
if (n(Ii(A))) {
G = 0 === p ? new Q(null, 3, 5, R, [ S, S, 1 ], null) : new Q(null, 3, 5, R, [ null, null, 1 ], null);
} else {
var I;
if (E.a("B", A)) {
I = new Q(null, 3, 5, R, [ jh, jh, E.a("B", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var K;
if (E.a("脟", A)) {
K = new Q(null, 3, 5, R, [ Lh, Lh, 1 ], null);
} else {
var O;
if (E.a("C", A)) {
O = Ni(q, p);
} else {
var V;
if (E.a("D", A)) {
var aa = q, Fa = p;
V = E.a("DG", Y(aa, Fa, 2)) ? n(Z(/^I|E|Y$/, Y(aa, Fa + 2, 1))) ? new Q(null, 3, 5, R, [ ph, ph, 3 ], null) : new Q(null, 3, 5, R, [ Eh, Eh, 2 ], null) : new Q(null, 3, 5, R, [ sh, sh, n(Z(/^D(T|D)$/, Y(aa, Fa, 2))) ? 2 : 1 ], null);
} else {
var Ja;
if (E.a("F", A)) {
Ja = new Q(null, 3, 5, R, [ eh, eh, E.a("F", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var J;
if (E.a("G", A)) {
J = Si(q, p);
} else {
var ef;
if (E.a("H", A)) {
var na = q, oa = p, sa = void 0, ta = void 0, ya = 0 === oa, ta = ya ? ya : Ii(Y(na, oa - 1, 1)), sa = n(ta) ? Ii(Y(na, oa + 1, 1)) : ta;
ef = n(sa) ? new Q(null, 3, 5, R, [ Gh, Gh, 2 ], null) : new Q(null, 3, 5, R, [ null, null, 1 ], null);
} else {
var za;
if (E.a("J", A)) {
var Aa = q, Ea = p, Ya = B, Ta;
if (E.a("JOSE", Y(Aa, Ea, 4)) || E.a("SAN ", Y(Aa, 0, 4))) {
var gb = Aa;
Ta = 0 === Ea && E.a(" ", Y(gb, Ea + 4, 1)) || E.a("SAN ", Y(gb, 0, 4)) ? new Q(null, 3, 5, R, [ Gh, Gh, 1 ], null) : new Q(null, 3, 5, R, [ ph, Gh, 1 ], null);
} else {
var Pa = Aa, Ua = Ea, Eb = Ya, yb = E.a("J", Y(Pa, Ua + 1, 1)) ? 2 : 1, Fb = void 0;
if (0 === Ua && Ld("JOSE", Y(Pa, Ua, 4))) {
Fb = new Q(null, 3, 5, R, [ ph, S, yb ], null);
} else {
var Pb = void 0;
var tc = Ii(Y(Pa, Ua - 1, 1));
if (n(tc)) {
var Uc = Ca(Hi(Pa)), Pb = Uc ? Z(/^A|O$/, Y(Pa, Ua + 1, 1)) : Uc;
} else {
Pb = tc;
}
var Vc;
if (n(Pb)) {
Vc = new Q(null, 3, 5, R, [ ph, Gh, yb ], null);
} else {
var pe = Pa, Rf = Ua, Sf = yb, Tf = void 0;
if (E.a(Eb, Rf)) {
Tf = new Q(null, 3, 5, R, [ ph, null, Sf ], null);
} else {
var Uf = void 0;
var Vf = void 0, Vh = /^L|T|K|S|N|M|B|Z$/, Wh = Y(pe, Rf + 1, 1), Vf = Ji.a ? Ji.a(Vh, Wh) : Ji.call(null, Vh, Wh);
if (n(Vf)) {
var Xh = /^S|K|L$/, Yh = Y(pe, Rf - 1, 1), Uf = Ji.a ? Ji.a(Xh, Yh) : Ji.call(null, Xh, Yh);
} else {
Uf = Vf;
}
Tf = n(Uf) ? new Q(null, 3, 5, R, [ ph, ph, Sf ], null) : new Q(null, 3, 5, R, [ null, null, Sf ], null);
}
Vc = Tf;
}
Fb = Vc;
}
Ta = Fb;
}
za = Ta;
} else {
var Wf;
if (E.a("K", A)) {
Wf = new Q(null, 3, 5, R, [ X, X, E.a("K", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var Xf;
if (E.a("L", A)) {
var Zh = q, $h = p, Oj = z, Pj = B;
Xf = E.a("L", Y(Zh, $h + 1, 1)) ? Ti(Zh, $h, Oj, Pj) : new Q(null, 3, 5, R, [ Tg, Tg, 1 ], null);
} else {
var Yf;
if (E.a("M", A)) {
var Zf = q, Oe = p, Qj = B;
Yf = E.a("UMB", Y(Zf, Oe - 1, 3)) && (E.a(Qj - 1, Oe) || E.a("ER", Y(Zf, Oe + 2, 2))) || E.a("M", Y(Zf, Oe + 1, 1)) ? new Q(null, 3, 5, R, [ Ug, Ug, 2 ], null) : new Q(null, 3, 5, R, [ Ug, Ug, 1 ], null);
} else {
var $f;
if (E.a("N", A)) {
$f = new Q(null, 3, 5, R, [ Nh, Nh, E.a("N", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var ag;
if (E.a("脩", A)) {
ag = new Q(null, 3, 5, R, [ Nh, Nh, 1 ], null);
} else {
var bg;
if (E.a("P", A)) {
var ai = q, bi = p;
bg = E.a("H", Y(ai, bi + 1, 1)) ? new Q(null, 3, 5, R, [ eh, eh, 2 ], null) : new Q(null, 3, 5, R, [ jh, jh, n(Z(/^P|B$/, Y(ai, bi + 1, 1))) ? 2 : 1 ], null);
} else {
var cg;
if (E.a("Q", A)) {
cg = new Q(null, 3, 5, R, [ X, X, E.a("Q", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var dg;
if (E.a("R", A)) {
dg = Ui(q, p, B);
} else {
var eg;
if (E.a("S", A)) {
eg = Wi(q, p, B);
} else {
var fg;
if (E.a("T", A)) {
var od = q, pd = p, gg;
if (E.a("TION", Y(od, pd, 4))) {
gg = new Q(null, 3, 5, R, [ Fh, Fh, 3 ], null);
} else {
var hg;
if (n(Z(/^T(IA|CH)$/, Y(od, pd, 3)))) {
hg = new Q(null, 3, 5, R, [ Fh, Fh, 3 ], null);
} else {
var ig;
if (E.a("TH", Y(od, pd, 2)) || E.a("TTH", Y(od, pd, 3))) {
var jg = od, kg = void 0;
var ci = Z(/^(O|A)M$/, Y(jg, pd + 2, 2));
if (n(ci)) {
kg = ci;
} else {
var di = Z(/^V(A|O)N /, Y(jg, 0, 4)), kg = n(di) ? di : E.a("SCH", Y(jg, 0, 3));
}
ig = n(kg) ? new Q(null, 3, 5, R, [ sh, sh, 2 ], null) : new Q(null, 3, 5, R, [ ih, sh, 2 ], null);
} else {
ig = new Q(null, 3, 5, R, [ sh, sh, n(Z(/^T|D$/, Y(od, pd + 1, 1))) ? 2 : 1 ], null);
}
hg = ig;
}
gg = hg;
}
fg = gg;
} else {
var lg;
if (E.a("V", A)) {
lg = new Q(null, 3, 5, R, [ eh, eh, E.a("V", Y(q, p + 1, 1)) ? 2 : 1 ], null);
} else {
var Pe;
if (E.a("W", A)) {
var qd = q, rd = p, Rj = B;
if (E.a("WR", Y(qd, rd, 2))) {
Pe = new Q(null, 3, 5, R, [ bh, bh, 2 ], null);
} else {
var mg, ng = qd, Qe = rd, og = void 0;
var ei = 0 === Qe;
if (ei) {
var fi = Ii(Y(ng, Qe + 1, 1)), og = n(fi) ? fi : E.a("WH", Y(ng, Qe, 2));
} else {
og = ei;
}
mg = n(og) ? new Q(null, 2, 5, R, [ "A", n(Ii(Y(ng, Qe + 1, 1))) ? "F" : "A" ], null) : new Q(null, 2, 5, R, [ null, null ], null);
var pg = L(mg, 0), qg = L(mg, 1), rg = void 0;
var sg = void 0, gi = E.a(Rj, rd), sg = gi ? Ii(Y(qd, rd - 1, 1)) : gi;
if (n(sg)) {
rg = sg;
} else {
var hi = E.a("SCH", Y(qd, 0, 3)), rg = hi ? hi : Z(/^EWSKI|EWSKY|OWSKI|OWSKY$/, Y(qd, rd - 1, 5));
}
var tg;
if (n(rg)) {
var Sj = qg;
tg = new Q(null, 3, 5, R, [ ud.b(pg), ud.b([ u(Sj), u("F") ].join("")), 1 ], null);
} else {
var ug;
if (n(Z(/^WI(C|T)Z$/, Y(qd, rd, 4)))) {
var Tj = qg;
ug = new Q(null, 3, 5, R, [ ud.b([ u(pg), u("TS") ].join("")), ud.b([ u(Tj), u("FX") ].join("")), 4 ], null);
} else {
ug = new Q(null, 3, 5, R, [ ud.b(pg), ud.b(qg), 1 ], null);
}
tg = ug;
}
Pe = tg;
}
} else {
var Re;
if (E.a("X", A)) {
var vg = q, Vd = p, Uj = B;
if (0 === Vd) {
Re = new Q(null, 3, 5, R, [ Lh, Lh, 1 ], null);
} else {
var ii = n(Z(/^C|X$/, Y(vg, Vd + 1, 1))) ? 2 : 1, wg = void 0;
var ji = E.a(Uj, Vd);
if (ji) {
var ki = Z(/^(I|E)AU$/, Y(vg, Vd - 3, 3)), wg = n(ki) ? ki : Z(/^(A|O)U$/, Y(vg, Vd - 2, 2));
} else {
wg = ji;
}
Re = Ca(wg) ? new Q(null, 3, 5, R, [ wh, wh, ii ], null) : new Q(null, 3, 5, R, [ null, null, ii ], null);
}
} else {
var Se;
if (E.a("Z", A)) {
var Wd = q, Xd = p;
if (E.a("H", Y(Wd, Xd + 1, 1))) {
Se = new Q(null, 3, 5, R, [ ph, ph, 2 ], null);
} else {
var li = E.a("Z", Y(Wd, Xd + 1, 1)) ? 2 : 1, xg = void 0;
var mi = Z(/^Z(O|I|A)$/, Y(Wd, Xd + 1, 2));
if (n(mi)) {
xg = mi;
} else {
var ni = Hi(Wd), xg = n(ni) ? 0 < Xd && Ld("T", Y(Wd, Xd - 1, 1)) : ni;
}
Se = n(xg) ? new Q(null, 3, 5, R, [ Lh, oh, li ], null) : new Q(null, 3, 5, R, [ Lh, Lh, li ], null);
}
} else {
Se = new Q(null, 3, 5, R, [ null, null, 1 ], null);
}
Re = Se;
}
Pe = Re;
}
lg = Pe;
}
fg = lg;
}
eg = fg;
}
dg = eg;
}
cg = dg;
}
bg = cg;
}
ag = bg;
}
$f = ag;
}
Yf = $f;
}
Xf = Yf;
}
Wf = Xf;
}
za = Wf;
}
ef = za;
}
J = ef;
}
Ja = J;
}
V = Ja;
}
O = V;
}
K = O;
}
I = K;
}
G = I;
}
m = G;
var oi = L(m, 0), pi = L(m, 1), Vj = L(m, 2) + g, Wj = null == oi ? h : Ec.a(h, oi), Xj = null == pi ? k : Ec.a(k, pi), g = Vj, h = Wj, k = Xj;
}
}
$ bin/uglifyjs -V
uglify-js 3.2.2
$ /usr/bin/time bin/uglifyjs repro.js -c | wc -c
0.41 real 0.49 user 0.03 sys
5252
md5-544f6843cd463ab7f5c536b73b91d16f
$ bin/uglifyjs -V
uglify-es 3.2.2
$ bin/uglifyjs repro.js -c
... runs for a very long time ...
I found the problem. PR in a sec.
kzc
on 18 Dec 2017
@kzc I can confirm that with the change from your PR, my full 12MB+ webpack js bundle now finishes uglifying. Thanks!
andreialecu
on 18 Dec 2017
Mangling aside, I suspected the code in the deeply nested reduced test case had to be generated - it originated from here:
https://github.com/Yomguithereal/clj-fuzzy/blob/master/src/clj_fuzzy/double_metaphone.cljc
It's more readable in the original form. :-)
kzc
on 18 Dec 2017
@andreialecu @kzc thank you both - great work! :wink:
alexlamsl
on 18 Dec 2017
Related issues
buu700
路
5Comments
pvdz
路
3Comments
alexlamsl
路
4Comments
hacdias
路
5Comments
diegocr
路
3Comments
Most helpful comment
I found the problem. PR in a sec.