Jest: Meta: Native support for ES Modules

[SimenB]

EDIT: quick guide for getting started: https://jestjs.io/docs/en/ecmascript-modules

ESM support will be unflagged in a future release of Node 12 (maybe not before April https://github.com/nodejs/node/pull/29866#issuecomment-574055057) and it is already unflagged in Node 13.2, so I think it's time to evaluate how we can add native support in Jest. I'll try to list which features Jest currently provides that are impacted by ESM support, and how we can solve/investigate them.

There is issue #4842, but I think that's more of a discussion issue, while this issue will be geared towards actually implementing support and more suitable to track for those who just want to get the current implementation status. Any comments added to this issue _not_ related to how we can implement support for the below enumerated features will be marked as spam - please direct any workarounds/discussions to separate issues. Also feel free to tell us if anything related to ESM features is missing from the list!

Please note that Jest will use the vm API (https://nodejs.org/api/vm.html) and as of writing (node v13.6) the ESM parts of this API is still flagged (--experimental-vm-modules). So saying ESM is unflagged is a bit of a misnomer at the moment. But I think we should start experimenting and potentially provide feedback to the Modules WG.

Lastly, I'm writing this issue mostly for people who will implement support, so it'll be somewhat low-level and specific to how Jest works. For people who _just_ want to know whether support has landed or not, I recommend using GH's wonderful "custom notification" and only subscribe to notifications on closing/reopening.

---

• [x] Running the module in the correct context

We achieve sandboxes by running a script within a given vm.Context (either provided by JSDOM or node core APIs). We need to do the same for ESM, but we'll need access to the context during construction of the module, not just when executing the module. I've opened up #9428 which adds the necessary APIs to JestEnvironment.

• [x] Globals

expect, test, beforeEach etc will still be added as globals, nothing should change here. jasmine global will also still be here.

• [x] jest "global" property

This is not really a global - it's injected into the module scope. Since the module scope is gone in ESM, we need to move it somewhere. Adding it to import.meta seems natural - there's an option called initializeImportMeta which we can use.

EDIT: Solution here is to fetch it via import {jest} from '@jest/globals'. We might still add it via import.meta in the future, but this should be enough for now.

• [ ] jest.(do|un)mock

Since ESM has different "stages" when evaluating a module, jest.mock will not work for static imports. It can work for dynamic imports though, so I think we just have to be clear in the docs about what it supports and what it doesn't.

jest.mock calls are hoisted, but that doesn't help in ESM. We might consider transforming import 'thing' to import('thing') which should allow hoisting to work, but then it's async. Using top-level await is probably a necessity for such an approach. I also think it's invasive enough to warrant a separate option. Something to discuss - we don't need to support everything jest.mock can for for an initial release.

• [ ] jest.requireActual

Not sure if how it should behave in ESM. Should we provide a jest.importActual and let requireActual evaluate in CJS always?

• [x] import.meta

Node has url as its only property (for now, at least). We need to make sure it's populated in Jest as well. We provide identifier instead of filename when constructing the module so I don't think it'll happen automatically, but url is essentially filename passed though pathToFileURL.

There's also an open PR for import.meta.resolve: https://github.com/nodejs/node/pull/31032

• [x] import thing from 'thing'

This should actually be fairly straightforward, we just need to implement a linker where we can also transform the source before returning it, meaning we don't need the loader API (which doesn't exist yet). This allows us to return mocks as well (albeit they'll have to come from a __mocks__ directory).

• [x] import('thing')

Essentially the same as above, but passed as importModuleDynamically when constructing the module. Will also support jest.mock, jest.resetModules etc more cleanly, so likely to be used quite a bit.

This can also be done for vm.Script via the same option.

• [ ] Handling errors during evaluation

Right now it's a runtime error (e.g. module not found), but that's not necessarily true with ESM. Does it matter for us? We should verify errors still look nice.

• [x] module.createRequire

We need to deal with this for people wanting to use CJS from ESM. I've opened up #9426 to track this separately as implementing it is not really related to ESM support.

EDIT: Implemented in #9469

• [ ] module.syncBuiltinESMExports

https://nodejs.org/api/modules.html#modules_module_syncbuiltinesmexports. Do we care about it, or is just making it a no-op enough? Not sure what the use case in Jest would be. Messing with the builtins is already breaking the sandbox and I don't think this should matter.

EDIT: #9469 made this into a no-op. I think that's fine?

• [ ] Detect if a file is supposed to be ESM or CJS mode

Inspecting type field in a module's package.json seems reasonable: https://nodejs.org/api/esm.html#esm_enabling. Should we also have our own config flag? Also needs to respect file endings.

https://github.com/nodejs/modules/issues/393

• [x] moduleNameMapper

Not sure if this impacts anything. I _think_ not since we'll be linking the modules together ourselves. Needs investigation, though.

EDIT: This is all resolution logic, which we control. So no changes here.

• [x] jest.config.mjs

Through #9291 we support jest.config.cjs - do we need to do anything special for .mjs? Probably use import('path/to/configFile.mjs') which means it'll have to be async. Is this an issue? Might be worth making config resolution async in Jest 25 so it's not a blocker for incremental support of ESM in Jest 25.

EDIT: #9431

• [ ] Package Exports

Node supports package exports, which sorta maps to Jest's moduleNameMapper, but also provides encapsulation features. Hopefully resolve will implement this, but if they do not we'll need to do something. Might be enough to use the pathFilter option? Unsure.

• [ ] JSON/WASM module

https://nodejs.org/api/esm.html#esm_experimental_json_modules. Do we need to care? Probably, especially for json. It's trivial for us to support import thing from './package.json' since we control the linking phase, but we probably shouldn't do it by default as it'll differ from default node. Should we force people to define a transform for it?

• [x] Code coverage

Does it matter? I don't think it's affected as we can still transform the source with babel (maybe it'll be confused by import statements, probably not) and V8 coverage definitely shouldn't care. We should verify though.

• [ ] Async code resolution

This is absolutely no blocker as sync resolution will work just fine. But we _can_ use async resolution now, which is great. I wonder if we should look into just using the resolve module off of npm again, as it already supports async. See #9505.

• [ ] Async code transformation

Similar to above, not blocking, but would be nice to support it. Might make @jest/transformer more usable in other environments as well. See #9504.

• [ ] Bad performance when accessing globals

Due to #5163 we have the extraGlobals option as a workaround - that workaround is no longer viable in ESM. I've opened up and issue with node here: https://github.com/nodejs/node/issues/31658

[SimenB]

I've landed very basic support with #9772. I've only tested the simplest cases, and there are many known limitations (most notably no jest object support and broken semantics when mixing CJS and ESM), but at least it's _something_. It'll go out in the next release of Jest (hopefully soon, only blocked by #9806)

[SimenB]

25.4.0 has been released with the first pieces of support. In addition to #9772 mentioned above, I've also included #9842. In _theory_ mixing CJS and ESM should work correctly now (🤞).

The one main missing feature is supporting the jest object. I haven't decided if we should stick it to import.meta or require people to import it through import {jest} from '@jest/globals'. Feedback appreciated!

I haven't written docs for this yet, but to activate it you need to do 3 things

1. make sure you don't run transform away import statements (set transform: {} in config or otherwise ensure babel doesn't transform the file to CJS, such as avoiding the modules option to preset-env)
2. Run node@^12.16.0 || >=13.2.0 with --experimental-vm-modules flag
3. Run your test with jest-environment-node or jest-environment-jsdom-sixteen

Please try it out and provide feedback! If reporting bugs, it'd be wonderful if you can also include how running the same code (minus any test specific code) runs in Node. I've read https://nodejs.org/api/esm.html _a lot_ over the last few weeks, but I've probably missed something.

[just-boris]

The one main missing feature is supporting the jest object. I haven't decided if we should stick it to import.meta or require people to import it through import {jest} from '@jest/globals'.

For the typescript use-case it is better to have an explicit import.

[SimenB]

Yup, I've added (and the temporarily reverted) a @jest/globals package that supports this, so that will be available regardless. I'm wondering if it makes sense to _also_ expose it on import.meta. Currently leaning towards not doing so, mainly since it's easier to add than remove later (and I'm personally no fan of globals)

[IlCallo]

+1 for the explicit import, it's a bit more verbose but simpler to understand

[zandaqo]

I'm getting this in Node 13.2 & Jest 25.4: ES Modules are only supported if your test environment has thegetVmContextfunction What am I missing?

[SimenB]

@zandaqo Oh sorry, forgot that point. Added above, but it's

Run your tests with jest-environment-node or jest-environment-jsdom-sixteen

[zandaqo]

ReferenceError: jest is not defined I'm guessing this is due to the missing @jest/globals

[SimenB]

Yes, as mentioned this will only work if you don't use the jest object.
Mocks are also probably broken, haven't tested them 😃

[s-ilya]

I've compiled a very basic project from what I see in e2e tests directory (e2e/native-esm/__tests__/native-esm.test.js) and in this issue. And unfortunately I still can't make it work 🙃Can anybody check it by any chance?

https://drive.google.com/file/d/1vyDZjsVKOTu6j55QA11GjO9E7kM5WX8_/view?usp=sharing

• [x] jest version 25.4.0
• [x] node version v13.12.0
• [x] package.json contains recommended jest options and no babel transformations seem to be in place

Running sample script (just importing double function and printing double(2)):

npm run main

> [email protected] main /Users/ilya/Projects/jest-esm
> node src/main.js

(node:16961) ExperimentalWarning: The ESM module loader is experimental.
4

Running jest with just one test for double function:

npm run test

> [email protected] test /Users/ilya/Projects/jest-esm
> jest

 FAIL  __tests__/native-esm.test.js
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/ilya/Projects/jest-esm/__tests__/native-esm.test.js:8
    import {double} from '../src/index.js';
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      at Runtime._execModule (node_modules/jest-runtime/build/index.js:1074:58)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.542s
Ran all test suites.

[SimenB]

You need to run node with --experimental-vm-modules and either name you file .mjs or "type": "module" in package.json.

EDIT: You probably have the latter seeing as it works outside of Jest for you, just missing the experimental flag to Node

[s-ilya]

@SimenB oh wow there are --experimental-vm-modules AND --experimental-modules. I was confused by the fact that --experimental-modules is not required starting from some node 13 version. Thanks!

TLDR: node --experimental-vm-modules node_modules/jest/bin/jest.js works for me

[SimenB]

Yeah, from the OP

Please note that Jest will use the vm API (https://nodejs.org/api/vm.html) and as of writing (node v13.6) the ESM parts of this API is still flagged (--experimental-vm-modules). So saying ESM is unflagged is a bit of a misnomer at the moment.

Awesome that it works for you, though!

(I'll mark these comments as resolved)

[aldeed]

@SimenB Thanks for this! Two issues I'm seeing so far.

Issue 1

• ESM unit test file imports default from ESM file (fn being tested)
• ESM file being tested imports default from a package, which exports CJS only
• Results in error: ReferenceError: module is not defined in the CJS package file

So I'm thinking this may not be correctly implemented?

An import statement can reference an ES module or a CommonJS module

This works fine when running the app. Named exports from CJS can only be imported by using createRequire, but default exports can just be imported.

Issue 2

When I'm not hitting the above error, I'm hitting this one:

TypeError: _vm(...).SyntheticModule is not a constructor

      at Runtime._importCoreModule (node_modules/jest-runtime/build/index.js:1198:12)

Project details

• Node 12.14.1
• Jest and babel-jest 25.4.0
• Latest Babel with targets: { node: 12 } and 2 plugins: babel-plugin-transform-import-meta and rewire-exports. (Tried removing import-meta plugin but got an error saying to add it back.)
• testEnvironment: "node" is pretty much the only config
• node --experimental-vm-modules node_modules/jest/bin/jest.js

If a reproduction repo would be helpful, let me know.

[SimenB]

Thanks @aldeed!

I'll look into issue 1, that indeed looks like a bug. EDIT: Should be fixed via #9850

Issue 2 needs node 12.16.0: https://nodejs.org/docs/latest-v12.x/api/vm.html#vm_class_vm_syntheticmodule

I'll change the check in Jest (right now it checks for vm.SourceTextModule which is available in more versions, but we need SyntheticModule as well).

[aldeed]

Confirmed that running with 12.16.0 fixes my issue 2. Will retest issue 1 after that fix is released. Otherwise waiting on the jest object for further testing, and I agree that it should need to be imported.

[beejunk]

Awesome work, @SimenB! I'm trying this out on a small project but ran into issues with dynamic imports. This is the error I'm seeing:

Module status must not be unlinked or linkingError [ERR_VM_MODULE_STATUS]: Module status must not be unlinked or linking

If I remove the dynamic imports then the test will run (but fail for other reasons, of course). The same test is currently working with Mocha (which shipped ESM support fairly recently).

If it helps, the dynamic imports in question can be seen here: https://github.com/beejunk/firn.js/blob/switch-to-jest/lib/renderPage.js#L43-L55

The test file is here: https://github.com/beejunk/firn.js/blob/switch-to-jest/test/build.test.js. The working Mocha version can be seen on the master branch.

Let me know if there's any other info that my be useful.

[SimenB]

Thanks @beejunk! I've been wondering if there could be race conditions between imports that import the same module before it's fully linked. It _seems_ like that's what your hitting here. I'll fix this today, thanks for the report!

EDIT: Fixed in #9858. Copied the fix over to your repo:

[dandv]

Has anyone managed to get this to work with TypeScript? node --experimental-vm-modules node_modules/jest/bin/jest.js returns the same SyntaxError: Cannot use import statement outside a module, even though my package.json does have "type": "module".

babel.config.cjs

module.exports = {
  presets: [
    '@babel/preset-typescript',
  ],
};

jest.config.cjs

module.exports = {
  testEnvironment: 'jest-environment-node',
  transform: {},
};

[SimenB]

@dandv You're essentially hitting this case: https://github.com/facebook/jest/pull/9772/files#r407255029

Could you open up a separate issue? Will need to figure out what to do with non-js extensions

[dandv]

@SimenB: #9860. Thanks for taking a look.

[SimenB]

@aldeed @beejunk 25.5.0 has been released with fixes for your issues. Keep the bug reports coming 😀

[SimenB]

Oh, additionally it includes support for import { jest } from '@jest/globals' for those of you waiting for that 👍

[beejunk]

Thanks for quick work on all this, @SimenB! I think I've run into another issue.

I've been experimenting with using query params in an import path as a way of busting the module cache on a development server. The idea is to have a file watcher that detects changes in a component and then updates the import path with an arbitrary query param so that the new code is immediately pulled in on the next page load without having to restart the server. This works when running the dev server. However, Jest is throwing the following error:

Cannot find module '/path/to/project/components/BasePage.js?cache=0' from 'renderPage.js'

Here's an example of where the query params are being used: https://github.com/beejunk/firn.js/blob/switch-to-jest/lib/renderPage.js#L55-L56

If I remove the query params then the tests will pass, although not consistently. If I run a single suite (for example, npm test -- test/build.test.js) the tests pass, but if I run all the tests at once they will fail most of the time with ambiguous errors. I'm still digging into the issue with the inconsistent tests, but I thought I'd report on the query param problem first.

[SimenB]

Thanks for the report @beejunk. #6282 is supposed to deal with this, but that also wants to pass the query to transformers and stuff, that we don't need here. So it might make sense to just support the query internally in the runtime for now, and let #6282 just deal with passing that query on.

I've added a bit of code to make the module cache vary by query: https://github.com/facebook/jest/blob/d425a49bd575e7167bc786f3c4f2833589091fa1/packages/jest-runtime/src/index.ts#L330-L334

But no code ever calls it with a query. I think we should be able to just do resolvePath.split('?') and all should work.

Regarding inconsistent errors, I'll take a look if that repo reproduces it. I haven't tested the ESM code with tests in parallel, only a single test. I'm unsure why it would impact things, but who knows 😀

[SimenB]

@beejunk query thing fixed in 25.5.1. I haven't had time to investigate the other issue yet

[nahueld]

I have a problem that I think may be related to this however didn't get fix in 25.X.

I'll try to summarise the escenario below:

• You have a test with a setup script
• That setup script will require a file dynamically, as in:
  { default: generator } = require(path.resolve(f))
• Everything inside f is not transpiled causing the "unexpected identifier import error".

This also happens if I try with import()

[SimenB]

Since you mention transpilation; if you have a setup that transpiles import to require this issue is not the correct place - this issue is about native support.

---

That said - you cannot require ESM, and I haven't been able to add support for import() from CJS yet due to Node having no API for it. Support for that has landed on Node master, but not released yet: https://github.com/nodejs/node/pull/32985. As can be seen in the linked release PRs, it'll come in 13.14.0 and 14.1.0. At that point I'll implement support for it 👍

You should be able to use a .mjs setup file though.

[aldeed]

@SimenB This is great, it seems to be working in several test files now! It's a bit of a process to resolve the differences between Babel and Node imports in each file, add jest import, etc., so I may come across more issues as I do that across hundreds of test files.

A few things that are more questions:

• what you said in your previous comment about support for import() from cjs, will that also allow for the Jest config file to be named jest.config.js? It currently only works for me when named jest.config.cjs and using module.exports = (error is TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified)
• The name "@jest/globals" fails the eslint rule node/no-extraneous-import due to not being a package listed in package-lock. Is there some reason for this naming convention? Is it possible to use jest without the @? It's easy to ignore the rule but I'm just wondering.
• Related question, why is jest global different from magic fns like test,describe, before, etc.? Should all of those be imported now, too?
• When this is finalized will it be possible for Jest to set --experimental-vm-modules flag? It kind of makes this seem nonstandard if it doesn't just work with the jest command. If you're not able to directly pass flags, possibly you can set/amend the NODE_OPTIONS env variable?
• I now seem to get a slightly different error than before when trying this on older Node, but neither error was clear that minimum Node version was the problem. Is it easy to add a check for minimum version with a more helpful error message?

[SimenB]

will that also allow for the Jest config file to be named jest.config.js?

It can be named .js if its closest package.json has type: 'module'. Otherwise to use ESM in the config file, it needs to be named .mjs. See https://nodejs.org/api/esm.html#esm_enabling. Note that the config file runs outside of Jest, so there we don't have much control. At the moment, we don't support async config, but awaiting an exported promise would be trivial, PR welcome 🙂 #8357 has stalled.

I'm very surprised you get A dynamic import callback was not specified though, we don't load the config file in a vm... Could you open up a separate issue?

The name "@jest/globals" fails the eslint rule node/no-extraneous-import due to not being a package listed in package-lock. Is there some reason for this naming convention? Is it possible to use jest without the @? It's easy to ignore the rule but I'm just wondering.

You should add a devDependency on @jest/globals. The package itself is solely type definitions. It's a separate package from jest so that type definitions work correctly, and so we can throw an error if it's loaded outside of the Jest runtime.

I don't currently have any plans to change that, but we could possibly deprecate that package and export the types from jest. That's a major breaking change though, so let's see later 🙂

Related question, why is jest global different from magic fns like test,describe, before, etc.? Should all of those be imported now, too?

jest is like require or module object from CJS in that it's unique per file, it's not actually a global (i.e. globalThis.jest === undefined). That allows jest.mock('../../file.js') etc to work correctly relative to the file it was injected in. You can choose to also import the actual globals, but they are still available as globalThis.expect etc.

When this is finalized will it be possible for Jest to set --experimental-vm-modules flag?

I think we'll wait for node to unflag them rather than silently set them - they're flagged for a reason, the API might change at some point. We could add an option that set NODE_OPTIONS I guess, not sure if that changes the current runtime? Regardless, no current plans for it.

I now seem to get a slightly different error than before when trying this on older Node, but neither error was clear that minimum Node version was the problem. Is it easy to add a check for minimum version with a more helpful error message?

Yeah, I'll be adding a better error message along with some documentation once the implementation stabilizes a bit. Seeing as the implementation is guarded by an experimental flag, I don't think anyone will stumble over it.

[beejunk]

@SimenB , I updated Jest to 25.5.2 and now all tests are passing. Query params are working and the intermittent errors I was seeing earlier are no longer happening. Thanks again for all your work!

[beejunk]

Okay, I saw the errors again on my last run of the tests, so that is still happening. I'll see if I can figure out a way to consistently reproduce and report back.

[beejunk]

I believe I've found a consistent way to reproduce the issue. Here is the branch I am working from: https://github.com/beejunk/firn.js/tree/switch-to-jest

To reproduce:

1. Delete the Jest cache if it exists. I just manually deleted /tmp/jest_rs.
2. Run npm test three times. The first two runs should succeed. However, the third run should fail.

This is the stack trace I'm seeing when the error does occur:

    Error: 
        at invariant (/home/brian/Projects/firn.js/node_modules/jest-runtime/build/index.js:1866:11)
        at Runtime.loadEsmModule (/home/brian/Projects/firn.js/node_modules/jest-runtime/build/index.js:480:7)
        at Runtime.linkModules (/home/brian/Projects/firn.js/node_modules/jest-runtime/build/index.js:548:19)
        at importModuleDynamicallyWrapper (internal/vm/module.js:397:21)
        at htmPreactPath (internal/process/esm_loader.js:31:14)
        at renderPage (/home/brian/Projects/firn.js/lib/renderPage.js:53:15)
        at map (/home/brian/Projects/firn.js/lib/build.js:43:12)
        at Array.map (<anonymous>)
        at build (/home/brian/Projects/firn.js/lib/build.js:36:43)
        at Object.<anonymous> (/home/brian/Projects/firn.js/test/build.test.js:57:5)

The error does seem to be originating with the dynamic imports. There's no error message, so I'm not entirely sure what is happening.

An additional note: If I clear the Jest cache and update the test script by adding --no-cache, then I am unable to reproduce the issue.

[SimenB]

Heh, I was lazy there and didn't provide a proper error message. The issue is that the test environment has been torn down, so I'm guessing there's a missing await somewhere. Didn't see anything looking through your code though, so I'll have to dig some more

[aldeed]

@SimenB Here is a reproduction of that ESM config issue: https://github.com/facebook/jest/issues/9935

[beejunk]

@SimenB I have created a minimal example for reproducing the Jest errors I mentioned above when using dynamic imports:

https://github.com/beejunk/jest-esm-dynamic-import-error

[SimenB]

Thanks for the great reproduction @beejunk! I've spent more hours that I care to admit here, without really understanding if it's a bug in Jest or in Node. I've reproduced the behavior using solely node core modules and reported it upstream, so let's see what they say: https://github.com/nodejs/node/issues/33216

[beejunk]

Thanks for looking into it, @SimenB. The tests seem to consistently pass if I add the --no-cache flag, which is fine for my use case. I appreciate all the work!

[SimenB]

Yeah, I noticed that as well. I think it's some sort of timing issue - with no cache things are slow enough that it works.

[aldeed]

@SimenB Thanks for resolving #9935. I mentioned a second concern in there, which I think is still valid. When type: "module", jest --init is still generating the config file with module.exports in it. This is relatively minor to change manually if you know what you're doing, but I think if ESM is unflagged in Node and lots of people start doing ESM projects, it will start to look more like a confusing bug (i.e., happy path of jest --init && jest on a new ESM project will throw an error). Should I submit a different issue specifically about improving the init logic?

[SimenB]

@aldeed are you sure? Testing right now gives me a mjs file with export default in it. I guess we could generate js and not mjs, but still. It uses ESM syntax

[aldeed]

@SimenB Well I was sure until you asked. 😄I tried it and you're correct. Maybe I initially did it with an older version of Node or Jest? Disregard.

[zandaqo]

This is awesome! Just reworked tests in one of my libraries to use ES Modules and ditched Babel. Thanks @SimenB!

Detect if a file is supposed to be ESM or CJS mode

Speaking of this, there are a lot of browser/bundler-oriented packages out their using "module":"<path to es module>" syntax to denote their ES module exports. It might be prudent to have some way of specifying how to resolve a given package regardless of the package's own settings. Something like moduleNameMapper but to specify if it's CJS or ESM.

[ahnpnl]

hi @SimenB , once this issue is closed, it means ts-jest can also unforce commonjs right ? Is file extension required to change from transformer side to work with esm ?

For example now ts-jest compiles ts to js for commonjs, does esm require file extension mjs when compiling from ts to js ?

[SimenB]

@zandaqo we won't support modules field, we'll be following node's spec and use exports: #9771. You can plug in your own resolver to support modules if you want, though: https://jestjs.io/docs/en/configuration#resolver-string. We might add some other option (mainFields, like webpack maybe?), but that'll come further down the line when the implementation stabilizes and we have less unknown unknowns 🙂

---

@ahnpnl #9860

[gabrieledarrigo]

Ciao guys!
Just a question: the blog post mentions that, since ES6 modules are static, they cannot be mocked; so, actually, there's no way to mock a module A imported by a module B in ES6?

[zandaqo]

@gabrieledarrigo I use moduleNameMapper for that, for example:

    "moduleNameMapper": {
      "moduleA": "<rootDir>/test/moduleA-mock.js"
    },

[SimenB]

@gabrieledarrigo you can do

jest.mock('the-thing-i-want-to-mock', () => /* do whatever in here */);

let importedThing;

beforeAll(async () => {
  importedThing = await import('thing-that-imports-mocked-module');
});

So you just need to make the import non-static and mocking will work.

I'm not sure it works now though, as I haven't wired up mock resolutions in the ESM code path yet. Will do so at some point soonish. But that is _probably_ going to be how we document module mocks for native ESM.

As mentioned in the blog pots, we'll be documenting these patterns at some point, we just have to figure them out 🙂

One idea we had was wait for top level await, then we could do this with a babel plugin.

[kuka-radovan]

@SimenB First of all, thank you for awesome job here :)

I actually face a problem when I would like to create customEnvironment extended from jest-environment-node. I need to import my server implementation there, which is written as esm. But it looks like, environment has to be defined as cjs.
My question is, is there an option to define custom testEnvironment as esm to be able to import my server module? Thank you for any advice.

[SimenB]

@kuka-radovan could you open up a separate feature request for that?

[aldeed]

UPDATE: This issue is now tracked in https://github.com/facebook/jest/issues/10025

@SimenB Thanks for the jest.mock advice above. As it happens I'm converting a few files that need that today. I can confirm that your example works when the mocked module is a node_modules package, but it isn't working for me mocking a module in the same project.

Here's a simple example:

// main.js
import secondary from "./secondary.js";

export default function main() {
  return secondary();
}

// secondary.js
export default function secondary() {
  return true;
}

// test.js
import { jest } from "@jest/globals";

jest.mock("./secondary.js");

let main;
let secondary;
beforeAll(async () => {
  ({ default: main } = await import("./main.js"));
  ({ default: secondary } = await import("./secondary.js"));
});

test("works", () => {
  secondary.mockReturnValueOnce(false); // TypeError: Cannot read property 'mockReturnValueOnce' of undefined
  expect(main()).toBe(false);
});

Exact same pattern works when "./secondary.js" is a package name instead. (I think the package I tried with exports CommonJS, if that matters.)

Jest 26.0.1 w/ Node 12.16.3

Any ideas, or should I submit a full separate issue?

EDIT: transform: {} in config, so no Babel at all

EDIT 2: This doesn't work either:

jest.mock("./secondary.js", () => ({
  default: jest.fn()
}));

[overlookmotel]

Amazing work on this.

However, unless I'm doing something wrong, it doesn't yet seem to be possible to use import() in a CJS test file.

I'm running Jest with node --experimental-vm-modules node_modules/jest/bin/jest.js and have testEnvironment: 'node', transform: {} in jest.config.js. This is on Node 14.2.0.

The import expression produces error:

TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]:
A dynamic import callback was not specified.

Is this a known limitation at present? I see https://github.com/nodejs/node/pull/32985 has now landed in Node 14.1.0.

[SimenB]

Yeah, I haven't gotten to implementing it yet. I'll probably land it this weekend.

@aldeed could you open up a separate issue? I need to go through and make sure mocks are part of the resolution, and your example seems like a good test case 🙂

[overlookmotel]

@SimenB Thanks for swift reply. I'll keep a look out for when it lands.

[SimenB]

Seeing as import in scripts might be reverted due to a regression (https://github.com/nodejs/node/issues/33166), let's hold off until that settles

[domenic]

I'm having issues trying to use this with .mjs test files. If I have __tests__/my-test.mjs, I get

$ yarn test
yarn run v1.22.4
$ node --experimental-vm-modules node_modules/jest/bin/jest.js
No tests found, exiting with code 1
Run with `--passWithNoTests` to exit with code 0
In C:\Users\Domenic\Dropbox\Programming\WIP\remember-to-eat
  1 file checked.
  testMatch: **/__tests__/**/*.[jt]s?(x), **/?(*.)+(spec|test).[tj]s?(x) - 0 matches
  testPathIgnorePatterns: \\node_modules\\ - 1 match
  testRegex:  - 0 matches
Pattern:  - 0 matches
error Command failed with exit code 1.

If I add

"testMatch": ["**/__tests__/**/*.mjs"]

to my package.json, I get

$ yarn test
yarn run v1.22.4
$ node --experimental-vm-modules node_modules/jest/bin/jest.js
No tests found, exiting with code 1
Run with `--passWithNoTests` to exit with code 0
In C:\Users\Domenic\Dropbox\Programming\WIP\remember-to-eat
  1 file checked.
  testMatch: **/__tests__/**/*.mjs - 0 matches
  testPathIgnorePatterns: \\node_modules\\ - 1 match
  testRegex:  - 0 matches
Pattern:  - 0 matches
error Command failed with exit code 1.

However if I remove the "testMatch" and then rename my file to __tests__/my-test.js, it works.

I'd like to be able to consistently use .mjs extensions in my project. Is that possible with Jest?

[overlookmotel]

@domenic I ran into this too. Solution is to add to config "moduleFileExtensions": ["js", "mjs"] (in addition to "testMatch").

[SimenB]

Took a look, and moduleFileExtensions is indeed necessary.

Jest gets a list of all files in the project by running hasteFS.getAllFiles() here:

https://github.com/facebook/jest/blob/2460c059ad1dbf124466ac25c8d5ccfd74ae9f25/packages/jest-core/src/SearchSource.ts#L159-L164

hasteFS is created as part of the HasteMap with the the following extensions config:

https://github.com/facebook/jest/blob/2460c059ad1dbf124466ac25c8d5ccfd74ae9f25/packages/jest-runtime/src/index.ts#L291

---

However, I don't think it should be necessary to specify moduleFileExtensions in this case though. We already force .snap to be found, should we force well-known JS extensions as well? Those being (off the top of my head) js, mjs, cjs, jsx, ts and tsx? It'll make the crawl slower, but I wouldn't think it has a huge impact. I might be wrong though? As a default it shouldn't be much slower since only cjs and mjs is not part of default patterns already, but for people who have custom patterns it might slow things down?

[domenic]

Yeah, it would be ideal if, at least in ES modules mode, .mjs just worked, without having to add moduleFileExtensions or modify the default testMatch.

It would also be nice if I could exclude .js files; when I tried that I got

 Validation Error:

  moduleFileExtensions must include 'js':
  but instead received:
    ["mjs"]
  Please change your configuration to include 'js'.

[SimenB]

I'm wondering if it makes sense to have some "ESM mode" which would add the node esm file extensions and also help with compile-to-js using esm to opt in (#9860).

---

Without js some stuff internally breaks that we load inside the sandbox (as it uses the same require implementation etc). We should probably fix that so the user cannot break us.

[aldeed]

Regarding slowing down, it's already quite slow on large projects, but I don't know that the number of extensions impacts that much. But I agree mjs and cjs should be added as defaults. Specifying moduleFileExtensions: ['js'] would override the defaults and speed it up, right? So maybe just document that as a performance tweak.

[franciscop]

Thanks for all of this work! It is certainly amazing. I followed the 3 steps ("type": "module" on my package.json, "testEnvironment": "jest-environment-node" in my jest config and --experimental-vm-modules on the CLI) and it seems too be working well 🎉

But then I'm trying to read and use import.meta as described in Node.js docs (and which seems to be already implemented judging from the checkbox) to create __dirname, but it seems like import.meta is failing:

console.log(import.meta);

SyntaxError: [PATH]/files.test.js: Support for the experimental syntax 'importMeta' isn't currently enabled (31:20):
    Add @babel/plugin-syntax-import-meta (https://git.io/vbKK6) to the 'plugins' section of your Babel config to enable parsing.

I don't have any babel and I thought babel was being left behind with this work. I'll come back to report if I can fix it somehow without installing babel.

Node.js v14.3.0, Jest v25.5.4

[franciscop]

I found a workaround for now. Since I am running the script from the same directory where my file is in my library, I can just do:

const __dirname = process.cwd();
const __filename = __dirname + "/files.test.js";

I'll follow this repo in case there's any update, thanks again for doing this!

[SimenB]

You need to explicitly opt-out of Babel by using transform: {} as config

[franciscop]

@SimenB I can confirm that adding transform: {} worked, thanks! I misunderstood that point as "don't add transforms" and not as "take transforms away" as intended.

BTW, testing has gone from 2.4 seconds to only 1.3 seconds and they consistently feel faster.

[SimenB]

Node 12 has been released with unflagged ESM (https://nodejs.org/en/blog/release/v12.17.0/). As noted in the OP though, the APIs Jest use are _not_ unflagged

[aledalgrande]

@SimenB I went over this thread several times but I am still stuck (using node 12.17).

When running Jest 26.0.1 I get this error:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /app/tests/setup.js
require() of ES modules is not supported.
require() of /app/tests/setup.js from /app/node_modules/@jest/transform/build/ScriptTransformer.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename setup.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /app/package.json.

I have transform: {}, and running with node --experimental-vm-modules node_modules/jest/bin/jest.js.

What am I missing?

[SimenB]

@aldarund not sure, could you put together a minimal reproduction?

[aledalgrande]

@SimenB here's a minimal repo to repro, just run yarn test https://github.com/aledalgrande/jest-example - I've tried with Node 13/14 and it's the same outcome. It seems to me that the flow for the global setup has not been updated to work with ESM.

also, you mentioned someone else 😆

[franciscop]

~Not @simenB, but @aledalgrande you seem to have everything correct here from what I tried, see my project fully running on ESM for comparison (jest config in the package.json).~

~To debug it if possible, I would suggest to simplify your jest config to _only_ have the two relevant properties, perhaps even in the package.json first. Then add each of the other properties you currently have to see which one works/didn't work.~

Ah the second comment mentions globalSetup and not normal tests, nvm my comment then. If I remove the globalSetup key in Jest, then the test runs as expected in that example, but the globalSetup key doesn't work as you said.

[SimenB]

Aha, I've forgotten about global setup and teardown. Can fix 👍

[franciscop]

Hi @SimenB , me again. Are named exports supported? With Node.js I can import and use a package like this:

import { customAlphabet } from "nanoid";

However, when trying to do a test that same code gives this error:

SyntaxError: The requested module 'nanoid' does not provide an export named 'customAlphabet'

For the tests, I can change the code to this and it works:

import nanoid from "nanoid";
const { customAlphabet } = nanoid;

But then the Node.js version stops working since there's actually no default sport (but for some reason the default export works with Jest):

SyntaxError: The requested module 'nanoid' does not provide an export named 'default'

The published (the repo seems to be in flux right now) nanoid code ends like this, with no default export:

export { nanoid, customAlphabet, customRandom, urlAlphabet, random }

[TrySound]

Jest consumes only "main" entry point. "exports" is not considered yet. You just import commonjs version which has only default export.

[franciscop]

Ah I see, the package.json seems to include this:

  "main": "index.cjs",
  "module": "index.js",
  "exports": {
    "./package.json": "./package.json",
    ".": {
      "require": "./index.cjs",
      "import": "./index.js",
      "browser": "./index.browser.js"
    },
    ...
  }
  ...

So probably Node.js is finding the module version, while Jest is using the CommonJS version that does not have a named export, right?

I'll wait until Package Exports is checked and then test it, thanks for all the work again! Marking these 2 comments as resolved until then. The test I'm referring to is this one.

[cyberwombat]

I'm revisiting this to see how its working - upgraded to Jest 26.0.1 and node 14.4. Set package.json to module type, set transform to {}, env to jest-environment-node and running with node --experimental-vm-modules. Now I get this new error:

ES Modules are only supported if your test environment has the `getVmContext` function

I have been unable to find info on this except a changelog from Jest saying getVmContext had been added a while back.

Any ideas?

[franciscop]

Could you share the relevant parts of your package.json please @cyberwombat ? Including the launching script that you are using for Jest.

[franciscop]

For reference, this is how it looks for me on a working project:

{
  ...
  "type": "module",
  "scripts": {
    ...
    "test": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
  },
  "jest": {
    "transform": {},
    "testEnvironment": "jest-environment-node"
  },
  ...

Then launch it with npm test

[cyberwombat]

@franciscop Mine basically is the same. Node 14.4.0. I can run yours fine. I will dive into things to see the diff.
package.json

{
  "type": "module",
  "devDependencies": {
    "jest": "^26.0.1",
  },
}

jest.config.js

export default {
  testEnvironment: 'jest-environment-node',
  setupFilesAfterEnv: ['./test/bootstrap.js'],
  testPathIgnorePatterns: ['<rootDir>/node_modules/', '<rootDir>/config/', '/<rootDir>/src/'],
  testRegex: '(\\.|/)(test|spec)\\.[jt]sx?$',
  transform: {
//    '^.+\\.jsx?$': 'babel-jest' // esm someday
  },
  transformIgnorePatterns: [],
  modulePaths: [
    '<rootDir>/test',
    '<rootDir>/src',
    '<rootDir>'
  ]
}

Script:
node --experimental-vm-modules node_modules/jest/bin/jest.js

[franciscop]

Not sure, but I'd try to work the other way around. Remove everything except transform: {} and testEnvironment: 'jest-environment-node', and start adding each of the options until you see which one triggers the previous error. I specially suspect transformIgnorePatterns _might_ be conflicting with transform, but I'm not that familiar with jest options.

[x80486]

Hello everyone! I ran into some issue while using Jest to test an Express application. More details here. Not sure if that's useful for what you are doing/tracking here :roll_eyes:

[franciscop]

@x80486 I encountered exactly the same issue yesterday. I've replied in StackOverflow with a longer explanation from my understanding.

Edit: I unhided my previous comment since it seems it might be relevant, this "exports" seems to be popular, very likely from this article on hybrid packages.

[SimenB]

exports is tracked in #9771

[cyberwombat]

@franciscop ok problem solved - it turns out there is a conflict in packages - I had serverless-bundle installed which causes the ES Modules are only supported if your test environment has thegetVmContextfunction error. I am not sure why - I would assume installing it would not cause a running conflict w Jest but evidently it does.

[ctavan]

@franciscop I think the reason why pkg.exports related issues start surfacing now is because that feature was unflagged in Node.js 14.x and some package maintainers (like me for uuid) started adding pkg.exports fields. So while you needed a commandline flag to activate that feature in Node.js 12.x you get that behavior by default now.

It will take a while for the whole ecosystem to adapt, so thanks for reporting issues around that topic!

[aldeed]

For those posting about exports, in case it has been lost in the long thread of this issue, my closed issue about it (https://github.com/facebook/jest/issues/9565) has an example of the moduleNameMapper workaround in it.

[akauppi]

The globalSetup problem reported in May is likely still there (Jest 26.1.0)? Getting the same errors as in the example repo @aledalgrande provides:

$ git clone [email protected]:aledalgrande/jest-example.git
$ cd jest-example
$ npm test

> @ test /Users/asko/Temp/jest-example
> node --experimental-vm-modules node_modules/jest/bin/jest.js --config=./jest.config.js

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/asko/Temp/jest-example/tests/setup.js
require() of ES modules is not supported.
require() of /Users/asko/Temp/jest-example/tests/setup.js from /Users/asko/Temp/jest-example/node_modules/@jest/transform/build/ScriptTransformer.js 

No rush. Checked CHANGELOG and it didn't mention a fix to globalSetup/globalTeardown with ES6.

Node.js 14.4.0, Jest 26.1.0

---

Update (13-Aug-20):

Still not possible, Node.js 14.7.0, Jest 26.4.0

[ahnpnl]

Side opinion but should this issue be a pinned issue since it’s the focus for jest at the moment ?

[Manish3323]

Any thoughts on what needs to be done to consume test reporters written in ES modules?...
with the latest jest version, i am getting error which essentialy says testScheduler expects custom reporter in commonjs format.

to see error

~/projects/esw-ts/lib/dist/test/testReporter.js:1

import os from 'os';

^^^^^^


SyntaxError: Cannot use import statement outside a module

    at wrapSafe (internal/modules/cjs/loader.js:1116:16)

    at Module._compile (internal/modules/cjs/loader.js:1164:27)

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1220:10)

    at Module.load (internal/modules/cjs/loader.js:1049:32)

    at Function.Module._load (internal/modules/cjs/loader.js:937:14)

    at Module.require (internal/modules/cjs/loader.js:1089:19)

    at require (internal/modules/cjs/helpers.js:73:18)

    at /Users/manish.gowardipe/Desktop/projects/esw-ts/lib/node_modules/@jest/core/build/TestScheduler.js:418:65

    at Array.forEach ()

    at TestScheduler._addCustomReporters (/Users/manish.gowardipe/Desktop/projects/esw-ts/lib/node_modules/@jest/core/build/TestScheduler.js:411:15)

[pepetorres1998]

Hi, I want to test Native support for ES Modules in my little project, but I'm new to NodeJS and I got lost in this Issue, I would love some guidance please.

• node --version: v14.5.0
• yarn jest --version: 26.1.0
• I'm trying to test this little project, is very simple.
• I have my files like this:

package.json

{
"jest": {
    "transform": {},
    "testEnvironment": "jest-environment-node"
  }
}

markov.test.js

const fs = require("fs");
const Markov = require("./markov.mjs");
// import fs from "fs";
// import Markov from "./markov.mjs";

const file = fs.readFileSync("text.txt", "utf8");
const markov = new Markov(file.toString());

test("Generates sentence with especified words", () => {
  expect(markov.makeSentence(8).length).toBe(8);
});

• I run yarn jest . and it gives me this error:
  

• I tried with node node_modules/jest/bin/jest.js . and it gives me the same error.

[cyberwombat]

@pepetorres1998 This thread is about running Jest with native esm modules which involves running things with certain flags/options - see the comment above for what to do (and set "type": "module" in package.json). Honestly though at this point it's not quite ready for prime time so if you are needing your project to work I might stick with Babel. There are a number of unchecked issues that are real show stoppers. I gleefully tried to switch a couple of weeks ago and came back crying to Babel.

[bdentino]

Is anyone else getting a ReferenceError: jest is not defined when trying to do things like jest.setTimeout(...) in a test file with this setup? Trying to figure out if this is related to es module environment, node version, jest version, or some combination of those things. (Currently using node v14.5.0, jest 26.1.0, environment jest-environment-node)

EDIT
I now see the unchecked checkbox in the issue description for the jest 'global' property. 🙃

[alexey-detr]

@bdentino I think you can try to import it explicitly import {jest} from '@jest/globals';

[Tzahile]

25.4.0 has been released with the first pieces of support. In addition to #9772 mentioned above, I've also included #9842. In _theory_ mixing CJS and ESM should work correctly now (🤞).

The one main missing feature is supporting the jest object. I haven't decided if we should stick it to import.meta or require people to import it through import {jest} from '@jest/globals'. Feedback appreciated!

I haven't written docs for this yet, but to activate it you need to do 3 things

1. make sure you don't run transform away import statements (set transform: {} in config or otherwise ensure babel doesn't transform the file to CJS, such as avoiding the modules option to preset-env)
2. Run node@^12.16.0 || >=13.2.0 with --experimental-vm-modules flag
3. Run your test with jest-environment-node or jest-environment-jsdom-sixteen

Please try it out and provide feedback! If reporting bugs, it'd be wonderful if you can also include how running the same code (minus any test specific code) runs in Node. I've read https://nodejs.org/api/esm.html _a lot_ over the last few weeks, but I've probably missed something.

@SimenB
This thread became enormous, and I think those who want to start with jest / use ES modules - will have difficulties finding and understanding the basic guidelines to start doing so.
Is there a formal explanation in the docs about adding jest to an ES-modules project (or some 'quick start')?

[guilhermetelles]

@aldeed Regarding your problem with mocking modules from the same project, did you found a fix? I'm having the exact same problem

(Btw, we also use reactioncommerce, so cheers to that haha)

[aldeed]

@guilhermetelles no, and it's tracked in https://github.com/facebook/jest/issues/10025 now.

[vvanpo]

I'm using Jest 26.1.0, node version 14.6.0 with --experimental-vm-modules, but I'm still seeing ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING when using import() inside of CommonJS. Should I try to come up with a minimal repro and open a new issue?

As an aside, is there an easy way to yarn link a copy of jest packages into a project now that Jest uses yarn berry? I wanted to try the latest master just in case this was implemented by not yet released. I was trying to do something like path/to/facebook/jest/.yarn/releases/yarn-sources.cjs link --all path/to/jest, but it would fail. Manually running something like cd node_modules; for p in jest*; do if [[ -d path/to/jest/packages/$p ]]; then rm -rf $p; ln -s path/to/jest/packages/$p; fi; done was not working either, I'm not sure why.

[SimenB]

@vvanpo import() in CJS was reverted in Node, you can follow https://github.com/nodejs/node/issues/31860

As for running local, I usually just uninstall jest from the project I wanna test and do ../jest/jest. Potentially nose ../jest/packages/jest/bin/jest.js. Just make sure to run yarn and yarn build:js first. If these instructions don't work (I'm writing from memory on a phone on a plane) please open up an issue (or PR) so we can properly write this into the CONTRIBUTING.md file

[mik-jozef]

Do you plan to support cyclic imports?

If I have a dummy test file that only imports one of two files that only import each other, I get RangeError: Maximum call stack size exceeded. If I remove one of the imports, the test passes. Repo that reproduces the issue.

[zsombro]

Hey! I set this up in an empty node project and it worked really well, however in our production setting, I get the following error message when I'm trying to run tests:

ES Modules are only supported if your test environment has the 'getVmContext' function

I saw someone else having the some problem in an earlier reply (by @cyberwombat ), but the package they found to be the culprit is not present in our package.json file. How to deduce the package (or setting) that causes the problem? I have tried systematically removing every jest setting that is not necessary to make this work, but I had no success.

UPDATE: I have managed to make progress by making a slight change in jest-runtime. I stopped the debugger at the line which tries to access the VM context and while the function really does not exist, this.context (which it should return) does, so I changed that line to access the property directly. I know this is probably not ideal, but maybe @SimenB this could give you an idea of what is going wrong?

Thank you in advance for any help

[SimenB]

Do you plan to support cyclic imports?

Definitely! Could you open up a separate issue?

---

@zsombro seems like you're running some old version of the test environment. If you run jest --show-config, what is displayed by testEnvironment?

[zsombro]

seems like you're running some old version of the test environment. If you run jest --show-config, what is displayed by testEnvironment?

@SimenB it says the following:

"testEnvironment": "/Users/zberki/git/project-name/node_modules/jest-environment-node/build/index.js",
"testEnvironmentOptions": {},

I just set it to jest-environment-node based on your instructions

Before I started this process, I upgraded jest using yarn add jest@latest. Do I have to upgrade the environment separately?

UPDATE: It turns out I had to. I deleted node_modules and yarn.lock to do a clean install and it still didn't work. However, if I manually add it by using yarn add -D jest-environment-node it seems to work. Is there a better way to manage this? I did a minimalistic test project before doing this on our codebase and I didn't have to do this

[SimenB]

yarn list jest-environemnt-node (or npm list jest-environemnt-node) will probably list multiple, is my guess

[zsombro]

├─ [email protected]
│  └─ [email protected]
└─ [email protected]

the 26.2.0 version is probably what I installed manually (at least based on package.json, which means jest-config has installed a version which is apparently outdated?

[SimenB]

You have something else pulling in an older version of jest-config (react-scripts perhaps (part of create-react-app)?). This issue is not the place to discuss it, tho 🙂

[akauppi]

Not being able to use ES modules in my globalSetup is beginning to hurt.

Two points:

• should this be mentioned as a checkbox in the beginning of this issue (so it gets tracked)?
• if there is an alpha/beta I could try, willing to do so

[wojtekmaj]

I:

• Ensured I'm running newest Jest version (26.4.0)
• Added jest-environment-node to my project
• Ensured it's not duplicated by inspecting the lockfile
• Added "testEnvironment": "jest-environment-node", in jest.config.json
• Added import { jest } from '@jest/globals'; wherever jest was used
• Ran test command setting --experimental-vm-modules by running them with NODE_OPTIONS='--experimental-vm-modules' yarn jest

And it crashes on the following code:

jest.mock('../../some/other/path', () => ({
  someOtherMethod: jest.fn().mockImplementation(…),
}));

with the following error (shortened - note "Allowed objects"!):

ReferenceError: src/foo/bar.spec.js: The module factory of `jest.mock()` is not allowed to reference any out-of-scope variables.
Invalid variable access: jest
Allowed objects: Array, …, jest, …, unescape.
Note: This is a precaution to guard against uninitialized mock variables. If it is ensured that the mock is required lazily, variable names prefixed with `mock` (case insensitive) are permitted.

I can't use Babel, because it improperly parses imports I fixed to run on Node 14 without Babel:

-import { map } from 'lodash';
+import lodash from 'lodash';
+const { map } = lodash;

Which unfortunately gets improperly parsed by @babel/preset-env, resulting inTypeError: Cannot destructure property 'map' of '_lodash.default' as it is undefined..

Can someone please help me work around this issue?

Edit: It seems like you _can_ use Jest+Babel on native ES modules-compatible code using CommonJS imports by doing this absolutely disgusting fix:

jest.mock('common-js-module', () => ({
  __esModule: false,
  ...jest.requireActual('common-js-module'),
}));

This way,

import lodash from 'lodash';
const { map } = lodash;

is perfectly consumed by Node 14, and the code resulting from running Jest+Babel,

var _lodash = _interopRequireDefault(require("lodash"));

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }

const {
  map
} = _lodash.default;

also runs.

[zsombro]

we've successfully converted all of our jest tests to use and import our ES6 code, but we got stuck on a few packages: namely puppeteer and uuid

The app only works if we import them into an object (like import uuid from 'uuid'), but the tests will not run this way. However, if we replace this import with the deconstruction syntax (such as import { v4 } from 'uuid', then it's the other way around: the test works, but the app throws an exception.

originally, we followed the guide and turned off every transformation, but we also tried creating a yarn workspace where we installed babel with a minimal node configuration, but this did not solve (or worsen) this specific problem

[jkrems]

However, if we replace this import with the deconstruction syntax (such as import { v4 } from 'uuid', then it's the other way around: the test works, but the app throws an exception.

That sounds like your app is compiled to CommonJS and isn't using modules in practice. From "real" ESM import uuid from 'uuid' shouldn't work because uuid has no default export and exposes an ESM build for node.

[grantcarthew]

Hey @SimenB, do you think some preliminary documentation on this would be a good idea?

[SimenB]

@grantcarthew definitely! I had hoped I'd be able to spend more time on this and stabilize it for Jest 27, but I doubt I'll be able to do that. But writing up a doc page about what's there now (and that it's experimental) sounds like a good idea

[dubzzz]

@SimenB I don't know what is the current status of the issue and if Jest should already work with my case or not but maybe it can help you somehow.

I'm trying to load a esm-only library (their extension is cjs but the type is module and node seems to be ok with that) but Jest fails to load it properly with error:

    C:\dev\codemirror-next-repro-cra\test-in-jest-esm\node_modules\style-mod\dist\style-mod.cjs:15
    export var StyleModule = function StyleModule(spec, options) {

Here the issue I originally opened https://github.com/codemirror/codemirror.next/issues/310. And a repro for Jest + ESM failing with node 14.13.1 https://github.com/dubzzz/codemirror-next-repro-cra/tree/main/test-in-jest-esm

[SimenB]

@dubzzz you can't have ESM in a cjs file. Node fails as well

$ node node_modules/style-mod/dist/style-mod.cjs
(node:48829) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/simen/repos/codemirror-next-repro-cra/test-in-jest-esm/node_modules/style-mod/dist/style-mod.cjs:15
export var StyleModule = function StyleModule(spec, options) {
^^^^^^

SyntaxError: Unexpected token 'export'
    at wrapSafe (internal/modules/cjs/loader.js:1172:16)
    at Module._compile (internal/modules/cjs/loader.js:1220:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)
    at Module.load (internal/modules/cjs/loader.js:1105:32)
    at Function.Module._load (internal/modules/cjs/loader.js:967:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47

[dubzzz]

Oups so sorry, I tried too quickly on node side. @nicolo-ribaudo already notified the author of the lib about this problem.
Really thanks a lot for your quick answer.

[SimenB]

I opened up a PR for some (pretty much stub) docs here: #10611. I didn't bother enumerating missing features/bugs as I think that'll be hard to keep in sync with reality, and looking at github issues is a better approach as they are (hopefully…) up to date.

[SimenB]

@Pomax as a new issue, please 🙂

[SimenB]

I just opened up #10620 which adds support for import() from CJS. Requested a few times are like https://github.com/facebook/jest/issues/9430#issuecomment-626054595

[ihoro]

Hello. It's quite hard for me to quickly embrace the whole story behind ESM in node/jest, so, probably, I'm asking something obvious or already answered. Do I understand it correctly that the following case is not supported yet? Or, I hope, I'm doing something not the correct way? I experience it like import x from 'x' works, but import { sub } from 'x' destructuring does not.

package.json:

{
  "name": "jest-uuid",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "test": "node --experimental-vm-modules node_modules/.bin/jest"
  },
  "devDependencies": {
    "jest": "26.5.2"
  },
  "dependencies": {
    "uuid": "8.3.1"
  }
}

f.spec.js

import { v4 } from 'uuid';
test('', () => {});

npm test

> npm test

> [email protected] test /Users/igoro/p/tmp/jest-uuid
> node --experimental-vm-modules node_modules/.bin/jest

 FAIL  ./f.spec.js
  ● Test suite failed to run

    SyntaxError: The requested module 'uuid' does not provide an export named 'v4'

      at jasmine2 (node_modules/jest-jasmine2/build/index.js:228:5)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.879 s
Ran all test suites.
(node:94492) ExperimentalWarning: VM Modules is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
npm ERR! Test failed.  See above for more details.

[SimenB]

You're waiting for #9771. Before that Jest doesn't know it's safe to load uuid as ESM (or rather, which file to load at which point it'd know it is ESM)

[Pomax]

Will this be following Node's own convention, where CJS can only be loaded as namespace, or will this "improve" on that by allowing syntax that doesn't actually work in Node itself? (e.g. Node does not allow import { readdirSync } from "fs-extra" because that's a CJS package, but it does allow import fs from "fs-extra"; which can then be unpacked using const { readdirSync } = fs).

[jkrems]

(e.g. Node does not allow import { spawn } from "child_process" because that's a CJS package, but it does allow import child_process from "child_process"; which can then be unpacked using const { spawn } = child_process;).

This is an unfortunate example because node considers "child_process" to be a "builtin" (and not CJS) module, so named exports do work. The latest nodejs also uses a heuristic to make many named exports work for CJS modules. That may be the hardest part to emulate.

[Pomax]

example updated to use fs-extra instead. But if named export are on Node's roadmap to land either this or next major, then Jest preempting that makes sense.

[SimenB]

That should already be implemented - Node core modules exposes named exports, "normal" CJS does not.

The latest nodejs also uses a heuristic to make many named exports work for CJS modules. That may be the hardest part to emulate.

Do you have a link to the PR implementing it? We should try to emulate it at least 🙂

[jkrems]

The PR is here: https://github.com/nodejs/node/pull/35249

The heuristic behind it is published as cjs-module-lexer (https://github.com/guybedford/cjs-module-lexer) but @guybedford might know more about any potential deviations.

[guybedford]

Just had a look at this and it appears like fs-extra is using an exports pattern like:

module.exports = {
  // Export promiseified graceful-fs:
  ...require('./fs'),
  // Export extra methods:
  ...require('./copy-sync'),
  ...require('./copy'),
  ...require('./empty'),
  ...require('./ensure'),
  ...require('./json'),
  ...require('./mkdirs'),
  ...require('./move-sync'),
  ...require('./move'),
  ...require('./output'),
  ...require('./path-exists'),
  ...require('./remove')
}

This is not currently a reexports analysis case we detect, but it might be possible to add to cjs-module-lexer if this would be a useful case to handle named exports detection for.

[SimenB]

Thanks @jkrems & @guybedford! I've opened up a PR now using that module: #10673

[guybedford]

The exact fs-extra support described in https://github.com/facebook/jest/issues/9430#issuecomment-713204282 is now implemented in [email protected], upstream tracking at https://github.com/nodejs/node/pull/35745.

_Update: testing out this build, it correctly detects all the fs-extra functions, but unfortunately it doesn't detect the Node.js native functions as they are not statically analyzable due to being populated by a for loop._

[trusktr]

feat: support named exports from CJS as named ESM imports #10673

I thought native ESM only supports importing a CommonJS module's exports as default?

[targos]

@trusktr not anymore: https://github.com/nodejs/node/pull/35249

[sdwlig]

Hello. It's quite hard for me to quickly embrace the whole story behind ESM in node/jest, so, probably, I'm asking something obvious or already answered. Do I understand it correctly that the following case is not supported yet? Or, I hope, I'm doing something not the correct way? I experience it like import x from 'x' works, but import { sub } from 'x' destructuring does not.

...
import { v4 } from 'uuid';

ESM modules do not support destructuring imports, even though the syntax looks like it does. For this to work, 'export v4' is needed. 'export default' will not match up.

https://kentcdodds.com/blog/misunderstanding-es6-modules-upgrading-babel-tears-and-a-solution

[TrySound]

@sdwlig uuid provides named exports and does not have default one. It should work but loading esm from packages with "exports" field is not supported by jest yet. Commonjs is loaded instead which is only available via default export.
https://github.com/uuidjs/uuid/blob/master/src/index.js

[conartist6]

Could we add package self-reference support (#10883) to this?