Playwright: [BUG] rimraf causes failure with multiple browsers on Windows
Context:
- PlayWright Version: 0.10.0
- Operating System: Windows 2019
Might relate to #680
Code Snippet
Run npx jest
const { chromium } = require("playwright");
test("browser one success", async () => {
const context = await chromium.launch();
await context.close();
});
test("browser two failure", async () => {
const context = await chromium.launch();
await context.close();
});
Describe the bug
The second test throws this error.
assert(received)
Expected value to be equal to:
true
Received:
false
at fixWinEPERM (node_modules/rimraf/rimraf.js:175:5)
at node_modules/rimraf/rimraf.js:160:15
Others have seen this same issue with rimraf on windows.
https://github.com/facebook/jest/issues/3942#issuecomment-313169747
https://github.com/GoogleChrome/lighthouse/pull/2641
I tried to force the playwright-core resolution to rimraf 3.0.1 but I ran into the same issue.
Is there a way to remove the rimraf dependency and replace it with fs-extra, that seems to be an approach that worked here. https://github.com/getgauge/taiko/issues/837
jperl
All 8 comments
I am happy to submit a PR if you are willing to accept replacing rimraf with fs-extra.
jperl
on 2 Feb 2020
I am happy to submit a PR if you are willing to accept replacing rimraf with fs-extra.
@jperl If I understand correctly, fs-extra simply includes rimraf:
So I wonder how is this fixing the problem. I can reproduce this easily on win - interestingly, it only happens under jest environment, and never otherwise.
The assert that fires is very peculiar too. Let me see what's going on!
aslushnikov
on 4 Feb 2020
@jperl so rimraf turned out to have an assert that makes sure that errors reported by fs module are, well, actually errors:
https://github.com/isaacs/rimraf/blob/d709272f6122b008de66650616fda34c8ae6f954/rimraf.js#L168
The assert fails because it looks like jest is messing up with fs errors somehow. I can repro this with the following jest test:
test('whoa', () => {
try {
require('fs').rmdirSync(__dirname);
} catch (e) {
console.log('caught error instanceof Error: ', e instanceof Error);;
}
});
Running npx jest prints "caught error instance of Error: false" - on both Mac and Windows to me.
This is somewhat surprising since jest explicitly states that they don't mock core node modules.
Let me dig further.
aslushnikov
on 4 Feb 2020
@aslushnikov Thank you for investigating this 🙏. I admittedly did not check that fs-extra solves this problem.
jperl
on 4 Feb 2020
Ok, we now know what's going on!
Investigation
- Jest runs tests in separate VMs (or execution contexts).
- For every VM, Jest creates a _test environment_: this defaults to JSDOMEnvironment. JSDOM's global is then used as VM's global.
- Test is executed in VM's global.
A reduced repro of this behavior looks like this:
const vm = require('vm');
const jsdom = require('jsdom');
const env = new jsdom.JSDOM('<!DOCTYPE html>', {
runScripts: 'dangerously',
});
vm.createContext(env.window);
const testcode = `
((console) => {
try {
non_existent_variable_that_throws_not_defined_error;
} catch (e) {
console.log(e);
console.log('caught error instanceof Error: ', e instanceof Error);;
}
})
`;
vm.runInContext(testcode, env.window)(console);
Reproduction without JSDOM
const vm = require('vm');
const context = { Error, console };
vm.createContext(context);
const testcode = `
try {
non_existent_variable_that_throws_not_defined_error;
} catch (e) {
console.log(e);
console.log('caught error instanceof Error: ', e instanceof Error);;
}
`;
vm.runInContext(testcode, context);
This code prints "false" because:
- the error in the global scope is coming from a parent execution context
- the error that we catch is generated by native c++ code and thus created by the child execution context
- these two errors are technically different in terms of
instanceof'ing.
Now, rimraf is doing an instanceof check for some of the errors it's getting from FS module:
https://github.com/isaacs/rimraf/blob/d709272f6122b008de66650616fda34c8ae6f954/rimraf.js#L168
This check fails and yields this error.
Mitigation
Unfortunately, configuring jest to run in a testEnvironment: node (and thus not bringing in JSDOM) doesn't help to mitigate this issue, because jest's NodeEnvironment is doing a very similar thing. (And there seem to be some workarounds for some similar problems)
Short-term, we should try fixing this upstream in rimraf - instead of instanceof'ing errors, it'd be probably fine to check for existence of a stack property.
Long-term, I wonder if there's something Jest can do better on their side to avoid this cross-vm type passing.
aslushnikov
on 4 Feb 2020
I'm impressed at how quickly you got to the root of this complex issue. Would you like me to submit the PR for rimraf?
jperl
on 4 Feb 2020
@jperl I'm on it, thanks! 👷♂️
aslushnikov
on 4 Feb 2020
Once/if https://github.com/isaacs/rimraf/pull/209 is merged and a new version is released, we can roll the dependency. It'll fix this bug.
We'll probably wait for a week to see if we can land the fix to rimraf. If there's a delay, we can detect jest environment and provide rimraf with custom fs hooks that re-throw vm-native errors
const isJestEnvironment = (function() {
try {
some_undefined_variable_goes_here;
} catch (e) {
return !(e instanceof Error);
}
})();
Related issues
jperl
·
4Comments
etnbrd
·
4Comments
wolf002
·
3Comments
jperl
·
3Comments
kblok
·
3Comments
Most helpful comment
Once/if https://github.com/isaacs/rimraf/pull/209 is merged and a new version is released, we can roll the dependency. It'll fix this bug.
We'll probably wait for a week to see if we can land the fix to rimraf. If there's a delay, we can detect jest environment and provide
rimrafwith custom fs hooks that re-throw vm-native errors