Jest: Jest globals differ from Node globals

Created on 10 Jan 2017  ·  63Comments  ·  Source: facebook/jest

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
After making a request with Node's http package, checking if one of the response headers is an instanceof Array fails because the Array class used inside http seems to differ from the one available in Jest's VM.

I specifically came across this when trying to use node-fetch in Jest to verify that cookies are set on particular HTTP responses. The set-cookie header hits this condition and fails to pass in Jest https://github.com/bitinn/node-fetch/blob/master/lib/headers.js#L38

This sounds like the same behavior reported in https://github.com/facebook/jest/issues/2048; re-opening per our discussion there.

If the current behavior is a bug, please provide the steps to reproduce and either a repl.it demo through https://repl.it/languages/jest or a minimal repository on GitHub that we can yarn install and yarn test.
https://github.com/thomas-huston-zocdoc/jest-fetch-array-bug

What is the expected behavior?
The global Array class instance in Jest should match that of Node's packages so type checks behave as expected.

I've submitted a PR to node-fetch switching from instanceof Array to Array.isArray to address the immediate issue, but the Jest behavior still seems unexpected and it took quite a while to track down.

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.
I am using the default Jest configuration (I have not changed any settings in my package.json).
Jest - 18.1.0
Node - 6.9.1 (also tested in 4.7.0 and saw the same error)
npm - 3.10.8
OS - Mac OS X 10.11.6

Bug Discussion Has Bounty

Most helpful comment

Is there any progress on this? This is leading to a very frustrating situation in one of my projects right now where a third party library is doing an instanceof check, and failing to recognize an array...

All 63 comments

This is likely due to the behavior of vm; see https://github.com/nodejs/node-v0.x-archive/issues/1277

Does Jest do anything to try to avoid this right now?

I came across the exact scenario. It's very hard to diagnose. I also came across it with Error objects. Writing wrapper workarounds for this is getting annoying.

From the linked nodejs issue:

Yes, Array.isArray() is the best way to test if something is an array.

However, Error.isError is not a function.

Jest team- should the node (and maybe jsdom) environment(s) be changed to put things like Error, Array, etc from the running context into the vm context? I believe that would solve this issue.

Alternatively, maybe babel-jest could transform instanceof calls against global bindings such that they work across contexts.

I don't like the babel-jest idea, if something like that is implemented it should be its own plugin. Other than that, I agree.

We can't pull in the data structures from the parent context because we want to sandbox every test. If you guys could enumerate the places where these foreign objects are coming from, we can wrap those places and emit the correct instances. For example, if setTimeout throws an error, then we can wrap that and re-throw with an Error from the vm context.

Is there any risk to the sandboxing added other than "if someone messes with these objects directly, it will affect other tests"? Or is there something inherent in the way the contexts are set up that would make this dangerous passively? Just trying to understand. I'd guess that instanceof Error checks are more likely than Error.foo = "bar" type stuff.

It's one of the guarantees of Jest that two tests cannot conflict with each other, so we cannot change it. The question is where you are getting your Error and Arrays from that are causing trouble.

They come from node native libraries like fs or http.

Ah, hmm, that's a good point. It works for primitives but not that well for errors or arrays :(

What if jest transformed instanceof Array and instanceof Error specifically into something like instanceof jest.__parentContextArray and instanceof jest.__parentContextError?

meh, I'm not sure I love that :(

We could override Symbol.hasInstance on the globals in the child context to also check their parent context if the first check fails... But Symbol.hasInstance only works in node 6.10.0+ or babel. Can't remember; does jest use babel everywhere by default?

I'm ok if this feature only works in newer versions of node. It seems much cleaner to me; assuming it doesn't have negative performance implications.

Assuming performance seems fine, which globals should it be applied to? Error and Array... Buffer maybe, too?

Yeah, that sounds like a good start.

I may be able to tackle a PR for this this weekend. I'm assuming we want it in both the node and jsdom environments?

I've started work on this in https://github.com/suchipi/jest/tree/instanceof_overrides, but am having difficulty reproducing the original issue. @PlasmaPower or @thomashuston do you have a minimal repro I could test against?

Not sure if it is 100% related or not but I have issues with exports not being considered Objects. For example the test in this gist will fail but if I run node index and log I get true: https://gist.github.com/joedynamite/b98494be21cd6d8ed0e328535c7df9d0

@joedynamite sounds like the same issue

Assuming performance seems fine, which globals should it be applied to? Error and Array... Buffer maybe, too?

Why not everything? I'm assuming performance won't be an issue as instanceof shouldn't be called often.

I ran into a related issue with Express+Supertest+Jest. The 'set-cookie' header comes in with all cookies in a single string rather than a string for each cookie. Here is a reproduction case with the output I'm seeing with Jest and with Mocha (it works with mocha): https://github.com/facebook/jest/issues/3547#issuecomment-302541653

Just spent a couple of hours trying to figure out what happened when an app failed in weird ways because of an instanceof Error check.

Basically, http errors seem to not be instances of Error, which is very frustrating.

Very simple, reproducible test case here.

I'm having trouble with http headers:
The following nodejs(8.9.1) code doesn't work in jest, I assume it has to do with an Array check?

const http = require('http');

const COOKIE = [ 'sess=fo; path=/; expires=Thu, 25 Jan 2018 02:09:07 GMT; httponly',
'sess.sig=bar; path=/; expires=Thu, 25 Jan 2018 02:09:07 GMT; httponly' ]

const server = http.createServer((req, res) => {
  res.setHeader('Set-Cookie', COOKIE);
  res.end();
});

server.listen(8000);

@t1bb4r I'm having the same issue, did you find a workaround for that?

@suchipi have you found any more time to be able to work on this? Would be amazing to solve this, and your idea of patching the instanceOf checks (which I didn't even know was possible, gotta ❤️ JS) seems like a really good solution.

I haven't looked at it in ages, but I still have a branch somewhere. I might be able to take a look this weekend.

Would love to get this into the next major!!

PR: #5995.

Please provide small reproductions of cases with different failing globals. The PR for now just handles Error (with 2 tests provided as reproductions in this issue), would love to have tests for Function, Symbol, Array etc as well

@SimenB small example with Array(simplified one):
```js
it.only('multiple parameters', function () {
let url = require('url');
let query = url.parse('https://www.rakuten.co.jp/?f=1&f=2&f=3', true).query;
console.log(query.f); // [ '1', '2', '3' ]
console.log(query.f instanceof Array); //false
});

btw i'm not fully understand why this labeled as "enchantment" instead of "bug" - Changing behaviour of instanceof for specific cases in the whole project that jest supposed to test, not looks like normal behaviour.

Is there any progress on this? This is leading to a very frustrating situation in one of my projects right now where a third party library is doing an instanceof check, and failing to recognize an array...

For the one willing to parse cookies with the extension cookie-session, a guy wrote a nice and easy solution on Medium.

Found a solution, we were having issues testing multiple set-cookie headers in node.js tests, this worked for us

Step 1
Create a testEnvironment file, we put it in utils/ArrayFixNodeEnvironment.js

Object.defineProperty(Array, Symbol.hasInstance, {
    value(target) {
        return Array.isArray(target)
    }
});

module.exports = require('jest-environment-node')

Step 2
Run jest with --testEnvironment flag

jest --testEnvironment ./utils/ArrayFixNodeEnvironment.js

@ssetem You're a lifesaver, thank you!

Hey folks, to get this moving forwards - we've decided to put a $599 bounty on this issue from our OpenCollective account.

If you're interested in doing some Open Source good-will and get paid for it, we're looking to have this shipped to production. There are a few ideas already e.g. https://github.com/facebook/jest/pull/5995 which you could take over, or you could start from fresh too

To get the bounty you would need to submit an expense via OpenCollective, here's their FAQ and I'll help that part of the process

@orta @SimenB I would love to work on what is left further. It would be great if you could give me the proper guidance as I'm not that aware of the context here :+1:

@jamesgeorge007 that's awesome! Not sure what information you're after - there's a bunch of reproducing examples in this issue (and linked) ones. You can also see my PR #5995 and take a look at fixing the errors from its CI run. I can rebase that PR now so it's fresh, though 🙂

Any questions in particular?

Unclear if we were hitting exactly the same bug - but upgrading the test environment to Node.js 10 resolved an issue for us where tests running under jest using supertest were failing to find the existing session, using multiple cookies.

That's expected, Node fixed their code to use Array.isArray. See https://github.com/facebook/jest/pull/5995#issuecomment-457840107

That's just a symptom though, the underlying issue is not solved

Found a solution, we were having issues testing multiple set-cookie headers in node.js tests, this worked for us

Step 1
Create a testEnvironment file, we put it in utils/ArrayFixNodeEnvironment.js

Object.defineProperty(Array, Symbol.hasInstance, {
    value(target) {
        return Array.isArray(target)
    }
});

module.exports = require('jest-environment-node')

Step 2
Run jest with --testEnvironment flag

jest --testEnvironment ./utils/ArrayFixNodeEnvironment.js

Unfortunately @ssetem's work-around does not seem to be working for ArrayBuffer on Node v8.x or Node v10.x

console.log('ArrayBufferFixNodeEnvironment');

// Fix for Jest in node v8.x and v10.x
Object.defineProperty(ArrayBuffer, Symbol.hasInstance, {
    value(inst) {
        return inst && inst.constructor && inst.constructor.name === 'ArrayBuffer';
    }
});

module.exports = require('jest-environment-node')

AFAICT it never calls the value function at all. I see it call print ArrayBufferFixNodeEnvironment when running so I'm fairly confident that the environment is being loaded.

jsdom here I come 🤷‍♂️

This issue also affects TypeError and RangeError. For example:

try {
    Buffer.alloc(0).readUInt8()
} catch (e) {
    console.log(e instanceof RangeError)
}

prints true when run in node, and false when run in jest. We had some type checks like this to determine whether to swallow exceptions and were quite confused when the tests failed.

I came across strange behaviour with classes as instanceof MyClass is also false in jest even though working completely fine when run outside jest.

Switching to ts and ts-jest solved the issue in tests. I didnt investigate what ts-jest does insside.

node 10.4
jest 24.10

I'm having the issue with fastify-static.
It depends on send, which itself depends on http-errors.

The later has an instanceof test which does not behave well in Jest:
https://github.com/jshttp/http-errors/blob/5a61a5b225463a890610b50888b14f16f518ac61/index.js#L56

function createError () {
  // so much arity going on ~_~
  var err
  var msg
  var status = 500
  var props = {}
  for (var i = 0; i < arguments.length; i++) {
    var arg = arguments[i]
    if (arg instanceof Error) {
      err = arg
      status = err.status || err.statusCode || status
      continue
    }

Until some of the PRs are merged here is the workaround (only solves instanceof Error case), all credits to @gkubisa:
.jest.json

{
  "setupFiles": [
    "./tools/tests/fix-instanceof.js"
  ]
}

tools/tests/fix-instanceof.js

'use strict'
const error = Error
const toString = Object.prototype.toString

const originalHasInstance = error[Symbol.hasInstance]

Object.defineProperty(error, Symbol.hasInstance, {
  value(potentialInstance) {
    return this === error
      ? toString.call(potentialInstance) === '[object Error]'
      : originalHasInstance.call(this, potentialInstance)
  }
})

Hi, what's the state? Is anybody working on this? How can we help to solve it?
it breaks my CI in the same way as described in https://github.com/facebook/jest/issues/2549#issuecomment-302543928 but only on the CI :confused:

This issue is so annoying I can't believe that this is unfixed for more than 2 years! Is everyone using a simple workaround which is not mentioned here and which renders this issue harmless? For me this issue is fatal. I have a large code base which works with all kinds of typed arrays and generic data transmissions for which instanceof checks are crucial. Up to now I used the Electron test runner which is not affected by this issue but now I also must make sure the code works in plain Node and for this Jest is totally broken.

Don't know if it helps (I read somewhere above that some people have trouble reproducing this?) but here is a test file for this problem which tests instanceof with various Node types (Array, Error, Promise, Uint8Array, Object):

const fs = require("fs");
const util = require("util");
const readFile = util.promisify(fs.readFile);

function getSuperClass(cls) {
    const prototype = Object.getPrototypeOf(cls.prototype);
    return prototype ? prototype.constructor : null;
}

describe("instanceof", () => {
    const buffers = fs.readdirSync(__dirname);
    const buffer = fs.readFileSync(__filename);
    const error = (() => { try { fs.readFileSync("/"); } catch (e) { return e; } })();
    const promise = readFile(__filename);

    const nodeErrorType = error.constructor;
    const nodeArrayType = buffers.constructor;
    const nodePromiseType = promise.constructor;
    const nodeUint8ArrayType = getSuperClass(buffer.constructor);
    const nodeTypedArrayType = getSuperClass(nodeUint8ArrayType);
    const nodeObjectType = getSuperClass(nodeTypedArrayType);

    const globalTypedArrayType = getSuperClass(Uint8Array);

    it("works with node array type", () => {
        expect(buffers instanceof Array).toBe(true);
        expect(buffers instanceof Object).toBe(true);
        expect([] instanceof nodeArrayType).toBe(true);
        expect([] instanceof nodeObjectType).toBe(true);
    });

    it("works with node error type", () => {
        expect(error instanceof Error).toBe(true);
        expect(error instanceof Object).toBe(true);
        expect(new Error() instanceof nodeErrorType).toBe(true);
        expect(new Error() instanceof nodeObjectType).toBe(true);
    });

    it("works with node promise type", () => {
        expect(promise instanceof Promise).toBe(true);
        expect(promise instanceof Object).toBe(true);
        expect(new Promise(resolve => resolve()) instanceof nodePromiseType).toBe(true);
        expect(new Promise(resolve => resolve()) instanceof nodeObjectType).toBe(true);
    });

    it("works with node Uint8Array type", () => {
        expect(buffer instanceof Buffer).toBe(true);
        expect(buffer instanceof Uint8Array).toBe(true);
        expect(buffer instanceof globalTypedArrayType).toBe(true);
        expect(buffer instanceof Object).toBe(true);
        expect(new Uint8Array([]) instanceof nodeUint8ArrayType).toBe(true);
        expect(new Uint8Array([]) instanceof nodeTypedArrayType).toBe(true);
        expect(new Uint8Array([]) instanceof nodeObjectType).toBe(true);
    });
});

The test creates instances of various types through the Node API and performs various instanceof checks against the global types and vice versa. All tests fail when Jest is using the Node environment. The tests work fine when using @jest-runner/electron instead or using Jasmine instead of Jest.

Maybe it is possible to disable the sandbox feature of Jest somehow? I consider this feature to be completely broken when simple stuff like fs.readFileSync(fileName) instanceof Object yields false instead of true.

This issue is so annoying I can't believe that this is unfixed for more than 2 years!

Feel free to contribute a fix. There's a reason there's a $499 bounty on the issue - it's far from trivial to solve.

Maybe it is possible to disable the sandbox feature of Jest somehow?

No, that's one of Jest's core features. However, you can do what the electron runner does, and create a custom environment and execute the script in the same context as the main node process: https://github.com/facebook-atom/jest-electron-runner/blob/d9546e4dd6bb9797dfc9e92d3288888564103cb5/packages/electron/src/Environment.js#L37-L41

It's fun because jest promotes

It works with projects using: Babel, TypeScript, Node, React, Angular, Vue and more!

but at the end they only support it's own (facebook) use cases. But anyway this is opensource :smile:

@kayahr I use a different test runner e.g ava for node.js only.
@SimenB good starting point I will try out as soon as I have time.

However, you can do what the electron runner does

Yes, I like this idea. I created this module now:

// SingleContextEnvironment.js
const NodeEnvironment = require("jest-environment-node");

module.exports = class extends NodeEnvironment {
    constructor(config, context) {
        super(config, context);
        this.global = global;
    }

    runScript(script) {
        return script.runInThisContext();
    }
}

And use it like this in my jest.config.js:

// jest.config.js
module.exports = {
    testEnvironment: "./SingleContextEnvironment",
};

Looks good so far, the tests are running now. Thanks.

No, that's one of Jest's core features.

Yes, I know, it's a core feature... But this core feature is broken. Sorry. It seems to work for many people because their projects have no need to use instanceof checks or they do not use the Node.js API in the tests at all, but for anyone else it is broken. So as long as this problem isn't fixed it would be nice to be able to simply disable this core feature with a configuration option instead of writing a custom environment implementation.

What about at least changing the classification of this issue from Enhancement to Bug?

@kayahr that was an easy job for $500 :rofl:

I agree it's a bug rather than an enhancement. 🙂 I recommend doing require.resolve("./SingleContextEnvironment") btw, just so jest doesn't have to guess what the path to the module is. Works way better in presets and other shared configs.

@StarpTech There's a _lot_ of people subscribed to this issue, please avoid spamming


For the record, the above environment breaks Jest's fake timers, and any modification you do to globals (removing, adding, changing) will leak between tests instead of tests running in isolation. This _might_ be an acceptable tradeoff for you and your project, but it is not the correct fix and will never be recommended.

Honestly I don't believe that this issue could be fixed by wrapping Node functions or overwriting instanceof checks. Workarounds like these just improve some situations but will never just work for anyone.

Type checking could also be done by using isPrototypeOf or comparing the constructor references for example instead of using instanceof and this would still be broken then and I don't see how the last one could be fixed with workarounds in Jest.

Or my project for example uses node-canvas so I have additional API calls which can produce objects and typed arrays. So when Jest goes the Wrap-all-the-Node-Functions way then I think it would not work for node-canvas or any other node extension out-of-the-box.

Modifying my code instead just to make it compatible to Jest also sounds like a terrible idea and may be even impossible because the affected code may be located in some third party framework library used by my application.

Was it already considered to run each test in a separate Node process when using the node environment? Wouldn't that be pretty much the same as running each test in a new electron window which works fine with the electron-runner? Don't know how badly this will affect performance, though...

However, you can do what the electron runner does

Yes, I like this idea. I created this module now:

// SingleContextEnvironment.js
const NodeEnvironment = require("jest-environment-node");

module.exports = class extends NodeEnvironment {
    constructor(config, context) {
        super(config, context);
        this.global = global;
    }

    runScript(script) {
        return script.runInThisContext();
    }
}

And use it like this in my jest.config.js:

// jest.config.js
module.exports = {
    testEnvironment: "./SingleContextEnvironment",
};

Looks good so far, the tests are running now. Thanks.

No, that's one of Jest's core features.

Yes, I know, it's a core feature... But this core feature is broken. Sorry. It seems to work for many people because their projects have no need to use instanceof checks or they do not use the Node.js API in the tests at all, but for anyone else it is broken. So as long as this problem isn't fixed it would be nice to be able to simply disable this core feature with a configuration option instead of writing a custom environment implementation.

What about at least changing the classification of this issue from _Enhancement_ to _Bug_?

When I do this I get several ● process.exit called with "0" on running tests. Any idea how to prevent this?

When I do this I get several ● process.exit called with "0" on running tests. Any idea how to prevent this?

I currently use this ugly workaround in the constructor of the SingleContextEnvironment to get rid of these messages:

// Make process.exit immutable to prevent Jest adding logging output to it
const realExit = global.process.exit;
Object.defineProperty(global.process, "exit", {
    get() { return realExit },
    set() {}
});

The logging in Jest comes from here: https://github.com/facebook/jest/blob/master/packages/jest-runner/src/runTest.ts#L201

So Jest overwrites the standard process.exit with a custom function which outputs this logging and there seems to be no clean way to prevent this. That's why I use the ugly workaround above which prevents setting a new value for process.exit.

I've hit this issue with process.emitWarning(), which does an instanceof Error warning check internally.

This works as expected outside Jest, but throws an ERR_INVALID_ARG_TYPE error in Jest:

global.process.emitWarning(new Error('message'))

I code native module (N-API) and use in one function instanceof checks (napi_instanceof) for differ Array & Map in arguments, but this check fail in my tests anytime.

napi_value global, Map;

napi_value iterable; // Map. Realy.

napi_get_global(env, &global);
napi_get_named_property(env, global, "Map", &Map);

bool isArray = false;
bool isMap = false;

napi_is_array(env, iterable, &isArray);
napi_instanceof(env, iterable, Map, &isMap)); // Pain

// isMap == false
// IsArray == false

if (isMap || isArray) {
    // Do my stuff never
}
else {
  // Forever
  napi_throw_error(env, NULL, "Arg must be Array or Map");
}

This fragment cannot be tested with Jest.

Ideas?

I thought this issue would no longer bother me since I use the SingleContextNodeEnvironment workaround which worked fine with Jest 24. But after upgrading to Jest 25 the workaround no longer works. When using it Jest complains that the describe function is not defined:

$ jest
 FAIL   node  src/test/instanceof.test.js
  ● Test suite failed to run

    ReferenceError: describe is not defined

       8 | }
       9 | 
    > 10 | describe("instanceof", () => {
         | ^
      11 |     const buffers = fs.readdirSync(__dirname);
      12 |     const buffer = fs.readFileSync(__filename);
      13 |     const error = (() => { try { fs.readFileSync("/"); } catch (e) { return e; } })();

      at Object.<anonymous> (src/test/instanceof.test.js:10:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.245s
Ran all test suites.

I published a small demo project (https://github.com/kayahr/jest-demo-2549) which can be used to reproduce this. Just run npm install and npm test. The project can also be used to reproduce the actual problem with the sandbox isolation by disabling the custom test environment in jest.config.js.

Well, any new ideas how to improve the workaround to get it working again with Jest 25?

Yeah, that broke due to the changes made to support V8 code coverage and future ESM support.
Might want to revisit the changes made, but for now you should be able to do delete NodeEnvironment.prototype.getVmContext or some such to restore jest 24 behaviour. Or just extend from jest-environment-node@24 which essentially does the same thing

~@kayahr I forked you example and I implemented @SimenB's advice. It seems to work!
Also, I implemented shared globals - which aren't working with current implementation, too.~

~Here the repo: https://github.com/balanza/jest-demo-2549~

thanks everybody

EDIT: the demo was actually misleading

@balanza it seems like your setup.js for globals is not working
And it should not, according to https://jestjs.io/docs/en/configuration#globalsetup-string

@balanza it seems like your setup.js for globals is not working
And it should not, according to https://jestjs.io/docs/en/configuration#globalsetup-string

Yes, you are right. That was built with a behavior in mind that's not actually what Jest is meant to do. Thanks for pointing out, I'll update my comment. Sorry if that made you waste some time 🤷‍♂

I'm using the SingleContextNodeEnvironment workaround in pretty much all of my projects now. Except the ones which use the electron runner which does not provide context isolation anyway and therefor is not affected by this annoying problem.

I have now centralized the workaround in a separate node module. Hopefully it's useful for others, too:

https://www.npmjs.com/package/jest-environment-node-single-context

Usage is very easy, see included README. But be aware that you no longer have context isolation when you use this environment so tests can have side effects on other tests by changing the global context.

I'm using the SingleContextNodeEnvironment workaround in pretty much all of my projects now. Except the ones which use the electron runner which does not provide context isolation anyway and therefor is not affected by this annoying problem.

I have now centralized the workaround in a separate node module. Hopefully it's useful for others, too:

https://www.npmjs.com/package/jest-environment-node-single-context

Usage is very easy, see included README. But be aware that you no longer have context isolation when you use this environment so tests can have side effects on other tests by changing the global context.

Can we include it in the readme somehow?

Was this page helpful?
0 / 5 - 0 ratings