Jest: coverage thresholds per file

Created on 29 Jun 2016 · 6Comments · Source: facebook/jest

measuring coverage for entire codebase might be not a good metric, but sometimes we want to ensure that critical parts of our app are covered.

I was thinking about something like

/* @jest-coverage-threshold: 90% */

describe('critical piece of the app', () => {
  // ...
});

would that be something we can do in jest?

Source

aaronabramov

👍3

Most helpful comment

is there a way to enforce something like no file should have less than 90% coverage?

it seems jest only supports global (i.e. aggregate) thresholds. say I have set global threshold to 90%, I have a hundred files most of them with 100% coverage but have some files with only 20% coverage, the test still passes cause overall coverage is above 90%

dalimian on 12 May 2017

👍20

All 6 comments

I believe there is quite a lot of impact to be had if you'd like to work on improving code coverage support in Jest and I'm happy to meet sometime this or next week to chat and also happy to put this on the H2 plan for Jest. Here are a few thoughts about this:

We've had a lot of discussions about this. Generally my team hasn't been excited about this idea so much and there are some concerns:

The coverage threshold is only going to be complete when we run all of the tests and collect aggregate results from all tests. When running only a single test file, coverage for a file might not be the same as when running multiple tests.
Apparently this was attempted in the past and people just don't care and lower the thresholds etc.
Coverage should consistently be high by default instead of having to add an annotation.
This actually won't affect code coverage at all.

However, there is a lot of stuff that we need to improve on around coverage, which is, besides mocking and matchers, another unloved part of Jest that we could easily fix and make awesome. Some of the bugs that I have on the list:

We need to merge coverage results per test result instead of at the end to reduce memory consumption in Jest. This is a problem right now.
Some things aren't properly covered through babel-jest + istanbul together. We could potentially use istanbul only for the reporting and use a babel plugin for gathering coverage. This would however leave behind people that don't use babel but given that this is what we use at Facebook and it would significantly improve our coverage implementation, I'd totally vote for doing it. We could also easily provide both modes (babel plugin when using babel-jest, istanbul when not using babel-jest).
We also need to improve caching of transformed files when using coverage. We'd get this for free when using a babel plugin to add the collectors.
jest -i --coverage is broken for some reason.

Another solution that I've been thinking about is to run every test twice on a diff (at FB) and compare the coverage percentage and comment on the diff if the threshold is lowered. Instead of running diffs twice, we could also store coverage info in a db, of course. Basically the way I see it is that coverage is an additional tool that helps during review and forcing people to write tests will probably make a lot of people angry. This is why we should the covered lines in phabricator diffs at FB which has been super useful for diff reviews.