Jest: Jest transform cache breaks tests using native code

[thymikee]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
Tests pass on first run or with --no-cache flag, otherwise one of them fails.

Repro
See following issues for details:

1. https://github.com/facebook/jest/issues/2029
2. https://github.com/facebook/jest/issues/3550

[jmurzy]

Do you have any pointers as to what might be causing this?

[alvinometric]

Same as @jmurzy I'd be curious to know, also is this definitely happening with all native packages ?

[jmurzy]

@alvinsight Looks like the issue with native modules is pretty much the same as #2826, nodejs/node#5016.

Also, when Object.defineProperty() is used to modify an existing property, it throws in watch mode unless you make it configurable—as it is immutable by default. I'm not sure what can be done in Jest to prevent this—perhaps rewrite all defineProperty declarations as configurable at test time using a babel transform. Not sure I like that though as it might cause unexpected side effects. Probably a better option is to add a global config for a blacklist of modules that should only be loaded once and only once. This would also "fix" the issues with native modules—at least the ones I'm experiencing so far.

@cpojer @thymikee Thoughts?

[SleepWalker]

Another one important thing needed for --no-cache workaround to work. The number of workers must be > 1, because the tests will still fail in runInBand mode, or when they are run in CI env with single core. So the test should be run with at least --maxWorkers=2.

[niba]

Adding --runInBand also breaks tests with native modules. The error

Exception has occurred: Error
Promise Rejection (TypeError: Cannot convert undefined or null to object)
TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at processExports (ROOT\test_libs\node_modules\promisify-node\index.js:44:16)
    at Object.<anonymous>.module.exports (ROOT\test_libs\node_modules\promisify-node\index.js:112:10)
    at Object.<anonymous> (ROOT\test_libs\node_modules\nodegit\dist\nodegit.js:131:26)
    at Runtime._execModule (ROOT\node_modules\jest-runtime\build\index.js:520:13)
    at Runtime.requireModule (ROOT\node_modules\jest-runtime\build\index.js:332:14)
    at Runtime.requireModuleOrMock (ROOT\node_modules\jest-runtime\build\index.js:408:19)

so it seems that it's impossible to debug tests with native.

What's worse I need to run my tests serially and right now I can't do it because of that error. Does somebody know any workaround to do it?

[MoOx]

Facing this issue. It's annoying to have to use --no-cache completely because of this :/
Hope a solution can be find.

[MoOx]

Note that in my case, no-cache is not helping everytime. I am trying --maxWorkers=2 --no-cache and now (with some updates in my code) it's impossible to have the tests to pass, always this error

TypeError: Cannot redefine property: length
        at Function.defineProperty (<anonymous>)

To reproduce:

git clone https://github.com/phenomic/phenomic.git
cd phenomic
git checkout 7adbf7dcac95f2f2de60e6111b97a3d06444d6fa
yarn
yarn tests

You should get something like this

$ jest --maxWorkers=2 --no-cache --coverage packages
 PASS  packages/core/src/db/__tests__/index-test.js
 PASS  packages/core/src/configuration/__tests__/flattenConfiguration.js
 PASS  packages/plugin-collector-files/src/__tests__/index.js
 PASS  packages/plugin-renderer-react/src/components/__tests__/Link.js
 PASS  packages/plugin-renderer-react/src/components/__tests__/BodyRenderer.js
  ● Console

    console.error node_modules/jest-mock/build/index.js:598
      @phenomic/plugin-renderer-react: BodyRenderer expects at least a child

 PASS  packages/plugin-renderer-react/src/__tests__/resolveURLs.js
 PASS  packages/plugin-transform-asciidoc/src/__tests__/index.js
 PASS  packages/core/src/api/__tests__/index.js
  ● Console

    console.error packages/core/src/api/index.js:255
      { NotFoundError: ID not found in database
          at new NotFoundError (...phenomic/packages/core/src/db/index.js:3567:177)
          at Object.get (...phenomic/packages/core/src/db/index.js:3625:13)
          at _callee9$ (...phenomic/packages/core/src/api/index.js:2265:25)
          at tryCatch (...phenomic/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:65:40)
          at Generator.invoke [as _invoke] (...phenomic/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:299:22)
          at Generator.prototype.(anonymous function) [as next] (...phenomic/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:117:21)
          at step (...phenomic/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
          at ...phenomic/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
          at new Promise (<anonymous>)
          at new F (...phenomic/node_modules/core-js/library/modules/_export.js:35:28)
          at ...phenomic/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
          at ...phenomic/packages/core/src/api/index.js:2293:21
          at Layer.handle [as handle_request] (...phenomic/node_modules/express/lib/router/layer.js:95:5)
          at next (...phenomic/node_modules/express/lib/router/route.js:137:13)
          at Route.dispatch (...phenomic/node_modules/express/lib/router/route.js:112:3)
          at Layer.handle [as handle_request] (...phenomic/node_modules/express/lib/router/layer.js:95:5)
          at ...phenomic/node_modules/express/lib/router/index.js:281:22
          at param (...phenomic/node_modules/express/lib/router/index.js:354:14)
          at param (...phenomic/node_modules/express/lib/router/index.js:365:14)
          at Function.process_params (...phenomic/node_modules/express/lib/router/index.js:410:3)
          at next (...phenomic/node_modules/express/lib/router/index.js:275:10)
          at expressInit (...phenomic/node_modules/express/lib/middleware/init.js:40:5)
          at Layer.handle [as handle_request] (...phenomic/node_modules/express/lib/router/layer.js:95:5)
          at trim_prefix (...phenomic/node_modules/express/lib/router/index.js:317:13)
          at ...phenomic/node_modules/express/lib/router/index.js:284:7
          at Function.process_params (...phenomic/node_modules/express/lib/router/index.js:335:12)
          at next (...phenomic/node_modules/express/lib/router/index.js:275:10)

***HUGE STACK TRACE, NOT IMPORTANT ***

PASS  packages/helpers-transform/src/__tests__/unifiedProcessor.js
PASS  packages/plugin-transform-json/src/__tests__/index.js
FAIL  packages/plugin-transform-markdown/src/__tests__/index.js

 ● Test suite failed to run

   TypeError: Cannot redefine property: length
       at Function.defineProperty (<anonymous>)



     at Object.<anonymous> (node_modules/oniguruma/src/oniguruma.js:97:8)
     at Object.<anonymous> (node_modules/first-mate/lib/grammar.js:10:10)
     at Object.<anonymous> (node_modules/first-mate/lib/grammar.js:389:4)
     at Object.<anonymous> (node_modules/first-mate/lib/grammar-registry.js:12:13)
     at Object.<anonymous> (node_modules/first-mate/lib/grammar-registry.js:273:4)
     at Object.<anonymous> (node_modules/first-mate/lib/first-mate.js:4:22)
     at Object.<anonymous> (node_modules/first-mate/lib/first-mate.js:8:4)
     at Object.<anonymous> (node_modules/highlights/lib/highlights.js:14:21)
     at Object.<anonymous> (node_modules/highlights/lib/highlights.js:429:4)
     at Object.<anonymous> (node_modules/remark-highlights/src/index.js:2:20)
     at Object.<anonymous> (packages/plugin-transform-markdown/src/default-options.js:63:3)
     at Object.<anonymous> (packages/plugin-transform-markdown/src/index.js:375:49)
     at Object.<anonymous> (packages/plugin-transform-markdown/src/__tests__/index.js:7:14)

PASS  packages/api-client/src/__tests__/url-test.js
PASS  packages/core/src/logger/__tests__/index.js
PASS  packages/plugin-bundler-webpack/src/__tests__/index.js
PASS  packages/cli/src/__tests__/check-engine.js
PASS  packages/plugin-renderer-react/src/shared/store/__tests__/index-test.js
...

You will see that errors come from oniguruma.js which have native modules.
Anything more I can do to help? I am totally unfamiliar with jest codebase etc.

[axiac]

With Nodegit, the --no-cache workaround doesn't help. It works with --maxWorkers with a value larger than the number of tests but this workaround reaches its limits easily.

Is there any hope this will be resolved soon?

[o2genum]

Unable to use Jest reliably to test an Electron app with native modules because of this.

Jest 24.8.0 and this kind of bug:

at debug (node_modules/ref/node_modules/debug/src/debug.js:65:17)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:746:3)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)
      at Object.writePointer [as _writePointer] (node_modules/ref/lib/ref.js:747:11)

[sbousamra]

I was unable to fix this issue with the suggested comments unfortunately, setting the workers worked sometimes, but other times didn't, not sure why.

I'm working around the issue by mocking the module that is causing the issue, which in my case is the 'ref-napi' sub-module of another module im using. Hopefully this could work for someone if that's possible in their use case as it was in mine.

Jest 23.5.0 and i get ```FAIL __tests__/unit/libs/test.ts
● Test suite failed to run

RangeError: Maximum call stack size exceeded

  at debug (node_modules/ref-napi/node_modules/debug/src/common.js:1:1)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:741:3)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)

[prurph]

I was unable to fix this issue with the suggested comments unfortunately, setting the workers worked sometimes, but other times didn't, not sure why.

I'm working around the issue by mocking the module that is causing the issue, which in my case is the 'ref-napi' sub-module of another module im using. Hopefully this could work for someone if that's possible in their use case as it was in mine.

Jest 23.5.0 and i get ```FAIL tests/unit/libs/test.ts
● Test suite failed to run

RangeError: Maximum call stack size exceeded

  at debug (node_modules/ref-napi/node_modules/debug/src/common.js:1:1)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:741:3)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)
  at Object.writePointer [as _writePointer] (node_modules/ref-napi/lib/ref.js:742:11)

@sbousamra I'm running into the same issue--how did you go about mocking the module? Thanks!

[sbousamra]

@prurph In my case I have a __ __mocks__ __ folder with mocks of certain modules I don't want to test, because I don't want my tests to fail if their module doesn't work for the purpose of unit tests. A mock of the file that was using ref-napi it looked something like this:

const someModule = jest.fn().mockImplementation(() => {
  return {
    init: jest.fn(),
    destroy: jest.fn(),
  };
});

module.exports = someModule;

Then wherever I use said module, I call jest.mock('someModule') at the top of that test file which means any test that subsequently will call that module, will run that mocked implementation. Obviously in your case, you need to change the contents of mockImplementation to suit the module your mocking, replacing it with functions that you call from that module, with jest.fn(). You can give it a more advanced mock implementation if required too.

Hopefully this helps, let me know if you have any more questions, happy to try assist :).

[prurph]

Cool--I get that, but what functions are you mocking in ref-napi such that it doesn't blow up? I tried doing no-op and naive implementations of .writePointer but then I get jest errors when it tries to create additional workers.

I was doing:

# __mocks__/ref-napi.js
const refNapi = jest.requireActual("ref-napi")

refNapi.writePointer = jest.fn()
refNapi._writePointer = jest.fn()

module.exports = refNapi;

but that just causes Jest to die creating more workers:

Call retries were exceeded at ChildProcessWorker.initialize

Also simply mocking init and destroy as you've shown blows up in files that require modules that have ref-napi as a dependency (argon2-ffi).

Do you mind sharing your actual mocked ref-napi implementation/override? Are you using the library directly or transitively?

[sbousamra]

Sorry I've re-read my post, and I realised since I made that post I changed my approach slightly haha. I'm not mocking ref-napi directly, I'm mocking the library I'm using that uses ref-napi. I tried mocking ref-napi too and ran into issues as well. Is it possible you're calling other functions of ref-napi in your project that you're not mocking there in your example? You might need to make sure you're not calling a single real function of theirs in your tests, and only mocks. I'm not sure what you're calling exactly, but that would be my first thing to check if it were me.

Mocking argon2-ffi should work though if I'm understanding correctly, as I've definitely been able to mock a module that has ref-napi as a dependency. My mock file looks almost identical to what I linked in the previous post, so possibly something else going on that might be causing the issue.

Sorry if that's not much help, if you can link more context I could try have a go at mocking it successfully for you, let me know if you make any progress.

[prurph]

To anyone else who finds themselves with this issue, I ended up stubbing out argon2-ffi, which is the library I was using that in turn requires ref-napi that in turn causes issues. My test case was simply asserting that stored passwords were hashed and that when retrieved they verified correctly, so the below "no-hash hash" is fine in tests.

// __mocks__/argon2-ffi.js
const hash = async function hash(buf, salt) {
  return Promise.resolve(buf);
};
module.exports = {
  argon2i: {
    verify: async (hashed, buf) => {
      return Promise.resolve(hashed === buf.toString());
    },
    hash,
  },
};

[andrewbranch]

I have a workaround that seems to work pretty well. It seems the issue only occurs when there are multiple test files that ultimately require the same problematic native module. Instead of letting Jest discover a bunch of files, you can give it a single entry point and require/import your files from there.

In my Jest config:

"testMatch": ["<rootDir>/test/index.js"]

test/index.js:

require('./one.test');
require('./two.test');
// ...

You could even glob for them and require in a loop if you have a lot of tests. One disadvantage is that snapshots all go to the same place. You also obviously can‘t filter a test run by file path anymore. This bug is a huge bummer.