Stryker: Running Stryker mutator 3.0.2 with Jest 25.2 fails all mutations

[doritrieur]

Summary

i have upgraded stryker to 3.0.2
using jest
when jest version was ^24.9.0", stryker run perfectly

when upgraded to the latest jest version (2.5.2) the stryker ran, but all mutations survived....

Stryker config

// eslint-disable-next-line func-names
module.exports = function (config) {
  config.set({
    mutator: 'javascript',
    packageManager: 'npm',
    reporters: ['html', 'progress', 'clear-text'],
    testRunner: 'jest',
    transpilers: [],
    coverageAnalysis: 'off',
    maxConcurrentTestRunners: 4,
    enableFindRelatedTests: true,
    mutate: ['src/**/*.js'],
  });
};

Test runner config
Jest

process.env.VERACODE_ID = 123;
process.env.VERACODE_SECRET_KEY = 123;
process.env.JENKINS_MS_URL = 'https://www.jenkins.ms:5555';

const config = {
  reporters: ['default'],
  coverageReporters: ['json', 'html', 'lcov', 'text'],
};
// if run on Jenkins
if (process.env.JENKINS_HOME) {
  config.reporters.push('jest-junit');

  process.env.RP_TOKEN = 'af888636-61d1-45c7-bfc9-223e3fc9cda3';
  process.env.RP_TAGS = process.env.RP_TAGS
    ? process.env.RP_TAGS
    : 'ms-veracode,unit-test';
  config.reporters.push([
    '@reportportal/reportportal-agent-jest',
    {
      endpoint: 'http://reportportal.intl.att.com:8080/api/v1',
      launchname: 'MS-Veracode__Unit-Test',
      project: 'scrumboard',
    },
  ]);
}
module.exports = config;

Stryker environment

 "@stryker-mutator/core": "^3.0.2",
    "@stryker-mutator/html-reporter": "^3.0.2",
    "@stryker-mutator/javascript-mutator": "^3.0.2",
    "@stryker-mutator/jest-runner": "^3.0.2",

 "jest": "^25.2.2",

Your Environment

| software         | version(s)
| ---------------- | -------
| node             |10.17.0
| npm              |6.11.3
| Operating System |
windows11

Add stryker.log

You can drag and drop it here. -->

// file can be downloaded, see #issuecomment-605462023

[kmdrGroch]

Could you please send generated file with logs instead of copy-paste? It will be more readable for us. Thank you :)

[doritrieur]

Hi sorry, i didnt know i can upload files.
attached summary and the logs below...

i have upgraded stryker to 3.0.2
using jest
when jest version was ^24.9.0", stryker run perfectly

when upgraded to the latest jest version (2.5.2) the stryker ran, but all mutations survived....

Stryker environment
"@stryker-mutator/core": "^3.0.2",
"@stryker-mutator/html-reporter": "^3.0.2",
"@stryker-mutator/javascript-mutator": "^3.0.2",
"@stryker-mutator/jest-runner": "^3.0.2",

"jest": "^25.2.2",

Your Environment

software version(s)
node 10.17.0
npm 6.11.3
Operating System
windows11

logstryker.txt
stryker.conf.txt
jest.config.txt

[nicojs]

I've been able to reproduce this issue with a simple jest example. I've added it as an e2e test and will now try to figure out what's wrong.

https://github.com/stryker-mutator/stryker/pull/2125

[nicojs]

As a workaround: set this in your Stryker config:

 "jest": {
    "enableFindRelatedTests": false
  }

[nicojs]

Since the build on Linux succeeds this is an issue on Windows only. See #2125. Will investigate more

[nicojs]

I've opened https://github.com/facebook/jest/issues/9728

Known workarounds (descending order of convenience):

• Renaming the .stryker-tmp directory to be a non-hidden directory.
  js // stryker.conf.[js/json] "tempDirName": "stryker-tmp",
• Disabling --findRelatedTests
js // stryker.conf.[js/json] "jest": { "enableFindRelatedTests": false }
• Use Jest@24
• Use Linux or WSL

[doritrieur]

OK, thanks, i will try the workarounds.

[kmdrGroch]

Is this issue still valid? there were some fixes and workarounds now.

[nicojs]

Yes, quite relevant. I want to keep this issue open for users looking for it. Let's hope someone can fix facebook/jest#9728