Jest: Ability to report coverage on untested files.

[trainiac]

One often wants to know the coverage of their codebase not just the files that are tested. For example, if I add a new file and don't write a test for it, that should be reflected in the code coverage.

What I think is needed is a more expressive way to say what files count as "the app". In the collectCoverageOnlyFrom config it would be great if you could say something like you would in gulp.

"collectCoverageOnlyFrom": ["src/**/*.jsx", "!__tests__/**/*.js"]  

(Everything that is a jsx file not under a tests directory)

[joestump]

It's not readily apparent, but I _think_ you might be able to achieve your stated goal by using testPathIgnorePatterns. Perhaps put __tests__ in there and see what happens. You'd think that Jest would ignore test files by default though.

NOTE: This also assumes Jest will not produce coverage reports for files matching testPathIgnorePatterns.

[trainiac]

@joestump thanks for your reply. My comment was a little misleading. I'm not concerned about ignoring files that I dont want tested. That can be achieved with testPathIgnorePatterns as you've said. It's when someone in a team adds a new file to a codebase that is not tested and is meant to be used in the application. Jest only runs coverage for files that are directly tested or files that are required by those that are directly tested.

[joestump]

I'm not sure this falls within the realm of Jest. Jest starts from __tests__ files and works its way backwards from there to create coverage files. I think what you're asking for is the reverse of that: look for all JS files and then see if they have a corresponding __tests__ file.

A simple BASH script in your CI pipeline to ensure every JS file had a corresponding __tests__ file according to your internal testing conventions should do the trick.

Do other tools do this? I don't think tooling I've used in Python or PHP have done this, but unsure about JavaScript. I'd be interested if others do offer such a thing as it sounds useful.

[trainiac]

Yeah a simple bash script could probably handle my issue.

I have written tests in python using py.test which has a plugin pytest-cov which does have this feature. Our would fail pull request builds if the number of uncovered lines went up. This was a really helpful enforcer for writing tests and inevitably raising code coverage.

[joestump]

Totally agree. We use py.test on some stuff as well; I'll need to check this feature out.

[ondrejhanslik]

I have to agree with the OP. I would actually go much further. My opinion is that jest takes a very complicated, unreliable, slow and uncofigurable approach to get the list of source files.

• Complicated

Most testing frameworks take a file pattern (e.g. ['src/**/*.js', !'src/**/*.spec.js']). That's easier to implement that parsing test files and looking for require. You can always exclude files with 0% coverage.

• Unreliable

For jest coverage to work correctly, you need to use require with static stings, you can't make a mistake and use import, you can't use some modules (e.g. require-all). It's too easy to make a simple mistake and still get a 100% coverage because the coverage for a file won't be calculated.

• Slow

Parsing the files to get the tree of dependencies is slow and it's probably the cause for a long pause after starting jest.

• Unconfigurable

The behavior cannot be changed and I am pretty sure that it's not what most people want. Most people want to know what parts of code are missing unit tests. Not what parts of code are already unit tested.

[screendriver]

+1

[ghost]

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

[james-hart]

+1

[giselermf]

+1

[ivantsov]

+1

[PedroPinheiro]

+1

[dmythro]

Any news on this? Very important feature to see the real coverage without hardcoding lists of files or workarounds.

[massick]

+1

[minhyeong-kim]

+1

[cpojer]

I'm going to close this because it is unlikely we'll add this to Jest any time soon. Of course, if anyone wants to build this, I'm very happy to reopen this and accept pull requests.

At Facebook we solve this kind of stuff during code review on a social level. We also do have a coverage reporter (in PHP that takes info from JS) that reports that the coverage of certain files is 0 % – however that knowledge hasn't generally been useful and we don't rely on it too much.

@ondrejhanslik I would recommend giving Jest another try. We have essentially rewritten Jest to be much better and easier to use, although some of the areas you mentioned we are still working on at the moment :)

[michaelsync]

@ondrejhanslik

The behavior cannot be changed and I am pretty sure that it's not what most people want. Most people want to know what parts of code are missing unit tests. Not what parts of code are already unit tested.

Please correct me if I misunderstand your explanation. If someone added a new code file without any test, it won't show in the test coverage report. That new file is missing unit tests. Is it not what people wants?

[aaronabramov]

I'll reopen it as i'm currently working on getting the coverage report for untested files.
before everyone used to have a require-all setup file that would just require every file in the codebase and add an empty coverage object to it, but that solution doesn't scale.

the solution i was thinking about is to have a separate step after the full test run + coverage that takes two arguments - existing generated goverage and list of all source files, and returns a modified coverage object with missing files added to it

[massick]

@dmitriiabramov it would be great and it's more or less what I wanted to do (but never did). Another weak point requiring all files is that requiring them is enough to have a coverage between 40/60% (depending on the complexity of the module) without writing any single line of test, while (imho) it should be 0.

[aaronabramov]

@massick exactly. although it makes sense that requiring files gives you 40% coverage, because you actually run that code when you require a file.
requiring all files at once is very slow though, and there's always going to be a module with a side effect that will ruin the whole test run :) so i'd really want to avoid doing it

[massick]

@dmitriiabramov yes I know, you are right and makes perfectly sense, that's why avoid requiring them but mark them as "not covered" can solve that "issue" :-)

[cpojer]

This is now part of the latest Jest pre-release and will be in Jest 15. @dmitriiabramov rewrote Jest's code coverage support. Please try with jest@test or [email protected] to see if this issue still persists. If it does, we'll reopen this issue.

The new option is collectCoverageFrom and it takes glob patterns. @dmitriiabramov can you make sure this is part of the API documentation before 15 comes out?

[aaronabramov]

@cpojer before i do that i wanted to do two things:

1. Make sure globs are fast enough
2. Add glob config options for other paths

[alansouzati]

I just had a chance to tet jest@test for this scenario and it works perfectly. thanks a lot guys!

[ms88privat]

Is there a way to ignore files with 0% coverage now?