Jest: Inconsistent output with --coverage

[azz]

🐛 Bug Report

When code-covered files exist in the root directory, the --coverage output will include the name of the directory. This is not desirable in CI scenarios where the working directory will vary.

Actual

https://github.com/azz/jest-istanbul-issue/tree/master

When a file exists at root (above), the output is:

$ jest --coverage
 PASS  ./index.test.js
 PASS  src/file.test.js
-------------------------|----------|----------|----------|----------|-------------------|
File                     |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-------------------------|----------|----------|----------|----------|-------------------|
All files                |      100 |      100 |      100 |      100 |                   |
 jest-istanbul-issue     |      100 |      100 |      100 |      100 |                   |
  index.js               |      100 |      100 |      100 |      100 |                   |
 jest-istanbul-issue/src |      100 |      100 |      100 |      100 |                   |
  file.js                |      100 |      100 |      100 |      100 |                   |
-------------------------|----------|----------|----------|----------|-------------------|

Test Suites: 2 passed, 2 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        1.983s
Ran all test suites.
Done in 2.98s.

Note the jest-istanbul-issue above.

Expected

https://github.com/azz/jest-istanbul-issue/tree/good-case

When it doesn't:

$ jest --coverage
 PASS  src/file.test.js
  √ it works (1ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |      100 |      100 |      100 |                   |
 file.js  |      100 |      100 |      100 |      100 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.708s
Ran all test suites.
Done in 2.76s.

[SimenB]

IMO the second case should include /src as well. So everything is always relative to the root of the project rootDir.

[SimenB]

@bcoe could you help out here? Is it possible to change the base reporting directory? I found https://github.com/gotwarlost/istanbul/blob/bc84c315271a5dd4d39bcefc5925cfb61a3d174a/lib/util/file-matcher.js, but setting a breakpoint in it doesn't seem to change anything.

Our code: https://github.com/facebook/jest/blob/22f67d49ffcce7a5b6d6891438b837b3b26ba9db/packages/jest-cli/src/reporters/coverage_reporter.js#L90-L104

I tried this, makes no difference:

diff --git i/packages/jest-cli/src/reporters/coverage_reporter.js w/packages/jest-cli/src/reporters/coverage_reporter.js
index 7d3fffe2a..9711378b5 100644
--- i/packages/jest-cli/src/reporters/coverage_reporter.js
+++ w/packages/jest-cli/src/reporters/coverage_reporter.js
@@ -21,7 +21,7 @@ import type {Context} from 'types/Context';
 import type {Test} from 'types/TestRunner';

 import {clearLine, isInteractive} from 'jest-util';
-import {createReporter} from 'istanbul-api';
+import {config as istanbulConfig, createReporter} from 'istanbul-api';
 import chalk from 'chalk';
 import istanbulCoverage from 'istanbul-lib-coverage';
 import libSourceMaps from 'istanbul-lib-source-maps';
@@ -87,7 +87,12 @@ export default class CoverageReporter extends BaseReporter {
       this._coverageMap,
     );

-    const reporter = createReporter();
+    // TODO: Figure out the common base of all of them
+    const root = Array.from(contexts)[0].config.rootDir;
+
+    const cfg = istanbulConfig.loadFile(null, {instrumentation: {root}});
+
+    const reporter = createReporter(cfg);
     try {
       if (this._globalConfig.coverageDirectory) {
         reporter.dir = this._globalConfig.coverageDirectory;

[flovilmart]

The reporter is text and the printed name make use of getRelativeName.
From reading in the implementation, this seems consistent with what is described in the issue (i.e. I don't believe there is a bug). The implementation of getRelativeName is referring to the parent level of the topmost name. Likely, this is an issue in the Istanbul reporter implementation and not in jest itself.

[SimenB]

@coreyfarrell thoughts on this one?

[coreyfarrell]

Probably would make sense to report relative to the detected/declared project root and also label that as "Project Root" instead of naming the FS directory. That said it's been this way forever and people tend to be pretty sensitive to reporting changes so I'm not sure I'd want to change this in a semver-minor. I'm otherwise occupied currently, it would help me if someone posted a bug to github.com/istanbuljs/istanbuljs/ and link to this issue.

[flovilmart]

In my opinion I don’t see this as a bug. The coverage will always report relative to the folder containing the topmost reported coverage, not related to the current working directory as initially reported.

[SimenB]

I agree it's not a bug in that it behaves as intended - I do think the behavior is not ideal, though.

I think we should be able to set a root, and that all paths should be relative to that regardless of what istanbul considers "top of the tree". If I do jest some-deep-test --coverage that only covers a subtree of files, check html coverage, run some other test that makes the tree different, a refresh of the html report should not show me the old file just because istanbul found a file higher up in the FS (or 404 if we had a cleanup between runs).

I can open up an issue tomorrow 👍