Jest: Coverage incomplete using `projects`

[robdonn]

Do you want to request a _feature_ or report a _bug_?

Bug

What is the current behavior?

If the current behavior is a bug, please provide the steps to reproduce and
either a repl.it demo through https://repl.it/languages/jest or a minimal
repository on GitHub that we can yarn install and yarn test.

• All tests run with their correct configuration
• A single coverage report is created at project root but the coverage is not complete
  
  
  
  • _products/product-2_ root index.js does not receive coverage, although the test covers it
  
  

Sample repo

--------------------------------|----------|----------|----------|----------|----------------|
File                            |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
--------------------------------|----------|----------|----------|----------|----------------|
All files                       |    85.71 |      100 |      100 |    85.71 |                |
 packages/package-1/src         |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 packages/package-1/src/example |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 packages/package-2             |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 packages/package-2/sample      |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 products/product-1             |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 products/product-1/sample      |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
 products/product-2             |        0 |      100 |      100 |        0 |                |
  index.js                      |        0 |      100 |      100 |        0 |              1 |
--------------------------------|----------|----------|----------|----------|----------------|

What is the expected behavior?

• All tests would run with their correct configurations
• A single coverage report would be created at project root

Please provide your exact Jest configuration and mention your Jest, node,
yarn/npm version and operating system.

Jest configurations are available in the sample repo. There is a configuration in the root of the repository and configurations also in each project folder for more project-specific stuff.

All coverage configuration is done at root as globalConfig as recommended in https://github.com/facebook/jest/issues/4255

MacOS High Sierra - 10.13.3
node - v9.3.0
yarn -v 1.3.2

[ranyitz]

Hey @robdonn
I've tried to recreate the issue locally.

1. Cloned to your sample project.
2. Ran yarn
3. Ran yarn test

Although I have matched versions of node, yarn, and even MacOs High Sierra I could not recreate it.

[rileylnapier]

i have this same problem too and its been driving me nuts.

possibly something to do with multiple packages that depend on each other using lerna? hmmm not sure

[DrewML]

Possibly related to https://github.com/facebook/jest/issues/6506

[wldcordeiro]

Still seeing this issue on latest, seems like repeated runs will eventually pick up the projects for me.

[SimenB]

Can someone set up a good reproduction case for this?

[DrewML]

@SimenB minimal test case is available in #6506. https://github.com/DrewML/jest-coverage-bug-repro

[nerdmax]

Same issue. It's driving me crazy. We have to depreciate the jest projects settings

[nrobertdehault]

@nerdmax I'm sure that's not optimal but running all the tests in parallel by passing --maxWorkers=100 fixed the coverage report for me.

[nerdmax]

@nrobertdehault Thank you for your reply. I tried --maxWorkers=100 but still not working 😟. I appreciate your suggestion.

[jure]

For what it's worth, we've had the same problem for a good while, but since we've been using the fix in https://github.com/istanbuljs/babel-plugin-istanbul/pull/171, we're getting coverage on all of our projects in a multi-project Jest... uhm... project. :) As described in the PR above, I believe this is the cause of several multi-project coverage related issues: this one, facebook/jest#5417 and facebook/jest#6483.

Give it a shot!

[SimenB]

That's awesome @jure!

[jorgenader]

Had similar issue with missing coverage in CI. Managed to locally repro this by running jest with --runInBand. Adding --maxWorkers=N fixed the problem in CI.

[panjiesw]

Got bitten by this recently as my test file count grows. The key to reproduce is to have many test files, far exceeding the default number of test workers.

All sample repro to similar issues to this have number of test files less than my cpu core, so I can't reproduce the issue (coverage generated correctly) with it.

My project has 23 test files across 5 sub-projects, running with default worker, the coverage is incomplete like this issue described. But running --maxWorkers 23 the coverage is generated correctly.

--no-cache and --runInBand doesn't make any difference for me.

[jure]

@panjiesw If you're using yarn, can you try using resolutions to do something like

  "resolutions": {
    "babel-plugin-istanbul": "https://github.com/jure/babel-plugin-istanbul.git#temp_release"
  }

The above uses the code from https://github.com/istanbuljs/babel-plugin-istanbul/pull/171, but compiled to lib/index.js. At your own risk etc, but this solved the issues for us and it would be good to get a confirmation.

[panjiesw]

@jure I'm not using yarn. Is there any similar way using npm to resolve babel-plugin-istanbul with your fix?

@SimenB are you still looking for good repro for this issue? There is one linked to this issue, if the author don't mind , which I have tried and also suffer from this issue: https://github.com/magento-research/pwa-studio

[jure]

@panjiesw For a simple one-off test, you could find the babel-plugin-istanbul dependency in your node_modules, and manually copy the contents of lib/index.js from https://github.com/jure/babel-plugin-istanbul/blob/temp_release/lib/index.js into it.

[panjiesw]

@jure got it working using your suggested solution. Can confirm now the coverage generated correctly, it works!

Now we wait until it's released and jest using the new babel-plugin-istanbul

[SimenB]

Jest uses v5 of babel-istanbul-plugin in the babel 7 branch (on hold due to another istanbul bug, see #7202). It will be part of Jest 24

[jure]

Just a heads up, the babel-plugin-istanbul that fixes the discussed issue hasn't been released yet, but the fix is merged to master.

[SimenB]

@DrewML your reproduction is fixed in Jest 24 (using [email protected]).

$ npx jest --version
24.0.0-alpha.12
$ npm run test:ohno

> [email protected] test:ohno /Users/simen/Development/jest-coverage-bug-repro
> jest --no-cache -i

 PASS   Project 1  packages/project1/index.spec.js
 PASS   Project 2  packages/project2/index.spec.js
-----------|----------|----------|----------|----------|-------------------|
File       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------|----------|----------|----------|----------|-------------------|
All files  |      100 |      100 |      100 |      100 |                   |
 project1  |      100 |      100 |      100 |      100 |                   |
  index.js |      100 |      100 |      100 |      100 |                   |
 project2  |      100 |      100 |      100 |      100 |                   |
  index.js |      100 |      100 |      100 |      100 |                   |
-----------|----------|----------|----------|----------|-------------------|

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        3.728s
Ran all test suites in 2 projects.

---

Gonna close, trying to find a reproduction to test the fix in babel-plugin-istanbul 😅

[DrewML]

Awesome - thanks @SimenB! Much appreciated.

[aj-anuragjain]

Adding these two configs will work for me.

"collectCoverage": true,
"collectCoverageFrom": ["/src//*/!(.spec).js"]