Stryker: Stryker fails to run with Jest + Babel + ES6

[mickeyvip]

Question

I would like to see a working example of Stryker with Jest and files written as ES6 modules.

I didn't find any such example in the documentation.

I am trying Stryker, but it fails with SyntaxError: Cannot use import statement outside a module.

Stryker environment

+-- @stryker-mutator/[email protected]
+-- @stryker-mutator/[email protected]

+-- [email protected]

Additional context

Seems like when Stryker creates the sandbox folder - Jest does not use Babel?

My files:

// src/myMath.js

export function add(a, b) {
  return a + b;
}

```js
// src/myMath.test.js

import { add } from "./myMath";

test("adds 1 + 2 to equal 3", () => {
expect(add(1, 2)).toBe(3);
});

```json
// package.json

{
  "name": "test-stryker",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest",
    "mutants": "stryker run"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@babel/core": "^7.12.10",
    "@babel/preset-env": "^7.12.11",
    "@stryker-mutator/core": "^4.2.0",
    "@stryker-mutator/jest-runner": "^4.2.0",
    "babel-jest": "^26.6.3",
    "jest": "^26.6.3"
  }
}

// babel.config.js

module.exports = {
  presets: [["@babel/preset-env", { targets: { node: "current" } }]],
};

// styker.conf.js

/**
 * @type {import('@stryker-mutator/api/core').StrykerOptions}
 */
module.exports = {
  packageManager: "npm",
  reporters: ["html", "progress"],
  testRunner: "jest",
  coverageAnalysis: "off",
  files: ["src/**/*.js"],
};

Jest output:

❯ npm run test                                                                                                                              

> [email protected] test /Users/me/Dev/Projects/test-stryker
> jest

 PASS  src/myMath.test.js
  ✓ adds 1 + 2 to equal 3 (1 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.158 s
Ran all test suites.

Stryker output:

❯ npm run mutants

> [email protected] mutants /Users/me/Dev/Projects/test-stryker
> stryker run

23:45:32 (62851) INFO ConfigReader Using stryker.conf.js
23:45:32 (62851) INFO InputFileResolver Found 1 of 2 file(s) to be mutated.
23:45:32 (62851) INFO Instrumenter Instrumented 1 source file(s) with 2 mutant(s)
23:45:32 (62851) INFO ConcurrencyTokenProvider Creating 11 test runner process(es).
23:45:32 (62851) INFO DryRunExecutor Starting initial test run. This may take a while.
23:45:34 (62851) ERROR DryRunExecutor One or more tests resulted in an error:
        undefinedCannot use import statement outside a module Jest encountered an unexpected token

This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

Here's what you can do:
 • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
 • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
 • If you need a custom transformation specify a "transform" option in your config.
 • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/en/configuration.html

Details:

/Users/me/Dev/Projects/test-stryker/.stryker-tmp/sandbox291806/src/myMath.test.js:2
import { add } from "./myMath";
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at new Script (vm.js:101:7)
    at Runtime.createScriptFromCode (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runtime/build/index.js:1350:14)
    at Runtime._execModule (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runtime/build/index.js:1245:25)
    at Runtime._loadModule (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runtime/build/index.js:898:12)
    at Runtime.requireModule (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runtime/build/index.js:746:10)
    at jasmine2 (/Users/me/Dev/Projects/test-stryker/node_modules/jest-jasmine2/build/index.js:230:13)
    at runTestInternal (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runner/build/runTest.js:380:22)
    at runTest (/Users/me/Dev/Projects/test-stryker/node_modules/jest-runner/build/runTest.js:472:34)
23:45:35 (62851) ERROR Stryker Error: Something went wrong in the initial test run
    at DryRunExecutor.validateResultCompleted (/Users/me/Dev/Projects/test-stryker/node_modules/@stryker-mutator/core/src/process/3-dry-run-executor.js:60:15)
    at DryRunExecutor.timeDryRun (/Users/me/Dev/Projects/test-stryker/node_modules/@stryker-mutator/core/src/process/3-dry-run-executor.js:70:14)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async DryRunExecutor.execute (/Users/me/Dev/Projects/test-stryker/node_modules/@stryker-mutator/core/src/process/3-dry-run-executor.js:33:42)
    at async Stryker.runMutationTest (/Users/me/Dev/Projects/test-stryker/node_modules/@stryker-mutator/core/src/stryker.js:36:53)
23:45:35 (62851) INFO Stryker Trouble figuring out what went wrong? Try `npx stryker run --fileLogLevel trace --logLevel debug` to get some more info.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] mutants: `stryker run`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] mutants script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/me/.npm/_logs/2020-12-21T21_45_35_623Z-debug.log

[simondel]

Hi! I may have misread it, but it looks like you have no Jest config specified anywhere, right?

I think that because of this, you'll have to use the config in the _Setup_ section of the babel-jest reader and add that to the jest section of your stryker config as documented here.

I hope that helps!

[simondel]

Oh and you may also want to copy your Babel config to the sandbox and other supporting files that are required to run your tests (copying the Babel config may cause Jest to magically start using it as well)

[mickeyvip]

@simondel , thank you.

I had jest entry in stryker.conf.js that pointed to jest.config.js that I had with the Babel option set. It didn't help.

The sandbox folder is created when Stryker runs, and it is Stryker that copies the files.

Seems like it does not copy babel.config.js.

I don't know how to tell Stryker copy additional files.

[simondel]

You can add the files you need (such as Babel an Jest configs) to files: ["src/**/*.js"],. If your files are in a git repo, you probably won't need the files config setting at all.

[mickeyvip]

@simondel thank you so much!

Adding the babel.config.js to "files" did the trick!

/**
 * @type {import('@stryker-mutator/api/core').StrykerOptions}
 */
module.exports = {
  packageManager: "npm",
  reporters: ["html", "progress"],
  testRunner: "jest",
  coverageAnalysis: "off",
  files: ["src/**/*.js", "babel.config.js"],
  jest: {
    configFile: "/jest.config.js",
  },
};