Jest: Add flag to ignore test suites that fail to run

[adanilev]

🚀 Feature Proposal

Add a flag that lets you ignore test suites (files) that failed to run instead of reporting them as failed.

It should not ignore tests that ran and failed.

Motivation

Make it easier to run one test suite and see the results when there are compile issues in the other test files that I want to ignore. I'm using TypeScript but suspect JS projects have the same behaviour.

When making a change that affects lots of files, I usually get the functionality working and then start fixing (and running) my unit tests one at a time. I run them one-by-one using the --testNamePattern= flag. The issue is that when you move/remove/rename a class, it breaks import statements in many tests and jest marks those tests as failed even if you're not targeting them. Same behaviour when you use --runTestsByPath.

This makes it difficult to see the results of the one test that you're interested in when there are 20 other failures.

Example

jest --ignoreFailedToRun -t="The test that I want to run"

Pitch

This would make jest easier to work with and the desired behaviour can't be achieved without changing jest (AFAIK).

[thymikee]

I'm not a fan, too easy to leave it in the npm script and you end up with evergreen tests.

[adanilev]

@thymikee, true, but the same could be said for the --testNamePattern= option, right? One could leave that in and ignore all/most tests. Perhaps it could give a nice, big warning if failures were encountered and ignored.

[mattphillips]

I agree with @thymikee I think this can allow false positives.

IMO this sounds like a UX problem, and Jest offers many options to target specific files already. The way I generally work with a large scale refactor that propagates to many files is I run the tests in watch mode and filter them with a file pattern using p and then a to run them all and find the next failing tests.

[laoshaw]

say I have 100 test cases, adding this flag can tell me 15 out of 100 is failing which is very useful, why can't jest do that? seems like a typical feature for other unit test harness, statistics matter when you have a large code base.

[aminya]

This absolutely needed.

Say, I have 600 tests and I know 3 of them will fail. But because I cannot tag those 3 tests for ignoring, I need to use a CI level flag such as continue-on-error: true to continue my CI on errors. This is dangerous since any error from those 600 can propagate while I only needed to ignore those 3.