Jest: Allow configuring using an array of `testRegex`

[jamietre]

🚀 Feature Proposal

Creating dynamic configurations to include or exclude specific things is difficult because of inconsistency in how config allows specifying inclusions and exclusions.

• testRegex allows only a single regexp
• testMatch allows an array of globs
• testPathIgnorePatterns allows an array of regexps

So there's no format (regex or glob) that can be used to build an array that works both for including and excluding.

It would be very convenient if either there were an analog to testMatch for ignoring (an array of globs), or it were permissible to provide an array of regexps instead of only a single one for testRegex.

I think the easiest solution to implement would be simply accepting an array or a single regexp for testRegex. Additionally, the regexp array convention is much widely used already (coveragePathIgnorePatterns, modulePathIgnorePatterns, transformPathIgnorePatterns, watchPathIgnorePatterns, testPathIgnorePatterns). Alternatively, a new config option testPathPatterns could be introduced that only accepts an array, for consistency with all the ignore options, but it seems redundant.

I'd like to take a shot at coding it, but wanted to vet the feature idea before doing any work.

Motivation

Motivation is creating simple configuration that can be used to run buckets of tests that are dynamically specified by pattern matching. For example, I would like to be able to specify patterns that match slow tests, and be able to produce dynamic config that can both exclude only slow tests, or run only slow tests.

Example

jest.config.js

const mode = process.env.JEST_MODE;
let testRegex = [/__tests__/\.tsx*$/];
const testPatternIgnorePaths = []'

const slowTests = /__tests__/.*\.slow\.tsx*$/;

switch(mode) {
   case 'slow':
       testRegex = [slowTests];
       break;
   case 'fast':
      testPatternIgnorePaths.push(slowTests);
      break;
   case default: // err
}

module.exports = {
  testRegex,
  testPatternIgnorePaths,
  ...
}

Pitch

Why does this feature belong in the Jest core platform?

It would be great for creating optimized configurations in large test environments. It makes the way file matching patterns are specified in configuration more consistent as well.

[SimenB]

@rickhanlonii @thymikee we just discussed config overhaul as it's quite the hot mess, and the mess that is regex vs glob, separate ignore fields etc should definitely be on the chopping block.

A single issue going through all config options (and CLI options) might make sense, although I guess file matching is big enough to be discussed on its own (e.g. in this issue).

---

So to be on topic; for file matching, of the top of my head, this is my suggestion: coveragePatterns: Array<Glob>, testPatterns: Array<Glob>, transformPatterns: Array<Glob> and remove everything else.

[jamietre]

Using globs exclusively is definitely preferable imho. I was going for a quick fix here; but I'm definitely in favor of standardization if breaking changes are on the table.

[rickhanlonii]

Short term I think it makes sense for testRegex to accept either a string or array 👍

[jamietre]

If you all are amenable to that solution for now I can take a crack at it, it doesn't seem like it would be a big change.

[SimenB]

Doing config.testRegex = Array.isArray(config.testRegex) ? config.testRegex : [config.testRegex]; in https://github.com/facebook/jest/blob/master/packages/jest-config/src/normalize.js should be easy enough to land, yeah

[rickhanlonii]

All you @jamietre!

[jamietre]

I have this working, except that in https://github.com/facebook/jest/blob/master/packages/jest-config/src/normalize.js line 348, we call validate which validates by matching types from an objectVALID_CONFIG: https://github.com/facebook/jest/blob/master/packages/jest-config/src/ValidConfig.js

There's no obvious way to make this work without adding features to the validator. I can think of lots of ways to make the validator handle multiple types (and indeed implemented one), but there are consequences beyond just accepting more than one option.

For example, when validation fails, it shows an example based on the ValidConfig object, and that would all have to be revised to support some syntax to show the multiple options correctly when there are errors. (Right now it just emits the key). This isn't undoable by any means but not sure it's a in the scope of what any of us were thinking as far as the impact this change would have. If you all are OK with changing the validator this way I'll finish it up, but this seems like it probably falls within the area that's under discussion for an overhaul anyway.

Other options I can think of are just skip validation for testRegex or add a new config option testRegexArray instead of overloading testRegex, and just map it in normalize to the internal testRegex. The second option seems to muddy the waters further though since you're looking at overhauling options processing anyway. Or maybe add it but leave it undocumented... what do you think.

[SimenB]

Mind pushing what you have to a branch so we cab play with it?

@thymikee thoughts on the validation?

[jamietre]

Here's the branch on my fork; lmk if I should make a pull request. (ignore launch.json changes - the default settings resulting in formatting being wrong every time I save!)

https://github.com/jamietre/jest/commits/jamiet/multiple-test-regexes

[thymikee]

I think I like the "MultipleValidOptions" addition to jest-validate, makes sense imho. Send a PR :)

[jamietre]

OK, sounds good... I still need to fix up how validation errors involving multiple valid options are reported, will do that & send it on.

[rubennorte]

Linking to #7185. This is something we'll probably want to include there