Uglifyjs: ufuzz failure

[alexlamsl]

// original code
// (beautified)
var _calls_ = 10, a = 100, b = 10, c = 0;

function f0() {
    {
        var brake1 = 5;
        L28953: while (typeof (1 === 1 ? a : b) && --brake1 > 0) {
            var expr2 = 38..toString();
            L28954: for (a in expr2) {
                4 in [ ,  ];
            }
        }
    }
}

var Math = f0(--b + {}, -1, {}.Infinity);

console.log(null, a, b, c, Infinity, NaN, undefined);

// uglified code
// (beautified)
var _calls_ = 10, a = 100, b = 10, c = 0;

function f0() {
    for (var brake1 = 5; 0 < --brake1; ) {
        var expr2 = 38..toString();
    }
}

var Math = f0(--b + {}, -1, {}.Infinity);

console.log(null, a, b, c, 1 / 0, NaN, void 0);



md5-ba84f17286be2ede890d2e8919875577



original result:
null '1' 9 0 Infinity NaN undefined

uglified result:
null 100 9 0 Infinity NaN undefined



md5-e6db36536cc0bb4bf1540520459a1890

minify(options):
{
"mangle": false
}

Suspicious compress options:
booleans
loops
side_effects
unused
```

[kzc]

The reduced test typeof (1 ? a : 0) is a very old bug and a good find. But it's a situation where reduce/test invented a test case unrelated to the original test case by dropping var declarations. The original ufuzz test case at the top of this issue was actually fixed by 93d084a1d191379a55498a12f79f573e7bb1d1be - not #4080. It doesn't matter in the end because both bugs were caught by fuzzing and fixed, but it's interesting. You can't really guide fuzzing. Had test/reduce not invented this bug test case it might not otherwise have been found by ufuzz.

$ git checkout c7a3e09407fbed928ccd874609c63c31b27f052a
HEAD is now at c7a3e09... enhance `loops` & `unused` (#4074)
$ bin/uglifyjs 4079.js -c | node
null 100 9 0 Infinity NaN undefined

$ git checkout 93d084a1d191379a55498a12f79f573e7bb1d1be
HEAD is now at 93d084a... fix corner case in `loops` & `unused` (#4076)
$ bin/uglifyjs 4079.js -c | node
null 1 9 0 Infinity NaN undefined

[alexlamsl]

Indeed − out of the 14 test failures fixed by #4076, there were two like this.

Almost tempted to reduce _successful_ test cases to see if we can generate more test failures :smirk:

[kzc]

Hmm... it's completely counter-intuitive, but that might actually work. Low success rate probably, but likely non-zero.

[kzc]

Upon reflection it would only work if the minified program produced different output from that of the original program. Introducing false original output wouldn't work either - the reduction process would be unconstrained and go off in a nonsensical tangent.

[alexlamsl]

It works for this test case with a slight modification to /test/reduce.js:

--- a/test/reduce.js
+++ b/test/reduce.js
@@ -30,6 +30,7 @@ module.exports = function reduce_test(testcase, minify_options, reduce_options)
     var verbose = reduce_options.verbose;
     var minify_options_json = JSON.stringify(minify_options, null, 2);
     var result_cache = Object.create(null);
+    result_cache[crypto.createHash("sha1").update(testcase).digest("base64")] = {};
     var test_for_diff = compare_run_code;
     // the initial timeout to assess the viability of the test case must be large
     var differs = test_for_diff(testcase, minify_options, result_cache, max_timeout);

```js
$ bin/uglifyjs 4079.js -c --reduce-test
// Node.js v14.7.0 on win32 x64
// reduce test pass 1, iteration 0: 411 bytes
// reduce test pass 1, iteration 25: 227 bytes
// reduce test pass 1, iteration 50: 154 bytes
// reduce test pass 1: 57 bytes
// reduce test pass 2: 14 bytes
// reduce test pass 3: 14 bytes
// (beautified)
typeof (1 ? a : 0);
// output: ReferenceError: a is not defined
// minify:
// options: {
// "compress": {},
// "mangle": false
// }

[kzc]

Interesting. You're using the original test case? Which git commit did you patch and are running against?

[alexlamsl]

You're using the original test case?

Yes.

Which git commit did you patch and are running against?

Just undo #4080 on top of master

[kzc]

We already knew it would emit that reduced test case with the typeof sequence bug present. But if you run 4079.js against latest master with the bug fixed it appears to emit the same unmodified program matching 4079.js with the original outputs after 16 seconds. So it does appear this proof of concept is workable for inputs not exhibiting a bug - at least for this particular test case.

Because your patch on latest master produces:

// output: [object Object]
// minify: null 1 9 0 Infinity NaN undefined

I'd suggest a slight modification along these lines to produce a hash as an output:

--- a/test/reduce.js
+++ b/test/reduce.js
@@ -32,2 +32,4 @@ module.exports = function reduce_test(testcase, minify_options, reduce_options)
     var result_cache = Object.create(null);
+    var hash = crypto.createHash("sha1").update(testcase).digest("base64");
+    result_cache[hash] = hash;
     var test_for_diff = compare_run_code;

and add some logic to compare the final minified output to the true original output.

The question remains whether pursuing this path is a profitable use of CPU time to try to reduce programs not exhibiting minify errors as compared to just generating new ufuzz inputs.

[alexlamsl]

That's why it was an one-liner − I'm trying to determine locally whether we have already exhausted the low-hanging fruits caused by ufuzz inability to generate certain patterns.

So far the statistics is not in this feature's favour 😅

[kzc]

Although ufuzz may not generate certain patterns there are enough false positives that it is effectively reducing many such cases anyway. But it's always good to challenge assumptions.