Jest: Jest doesn't always try to run test suites in parallel

[gsteacy]

Do you want to request a _feature_ or report a _bug_?

Bug

What is the current behavior?

Repro here: https://github.com/gsteacy/jest-bug-repro. Running npm test is sufficient. Ignore the test failure. I'm using the same repro for multiple bugs.

The problem (assuming this is not by design) is not consistently reproducible. For the repro given, Jest runs both test suites in parallel when the cache is ignored with --no-cache or if there is no cache available. Subsequent runs with a cache run the test suites serially.

For some larger projects I have (with more complex configurations) Jest will always run the tests in parallel unless told to do otherwise, but for other projects it won't. I haven't been able to find a pattern.

What is the expected behavior?

Jest runs the test suites in parallel as --runInBand has not been specified.

Please provide your exact Jest configuration

Defaults

Run npx envinfo --preset jest in your project directory and paste the
results here

  System:
    OS: Windows 8.1
    CPU: x64 Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
  Binaries:
    Node: 6.13.1
    Yarn: 1.3.2
    npm: 3.10.10
  npmPackages:
    jest:
      wanted: ^22.4.2
      installed: 22.4.2

I have also tried it with Node 8 and there is no apparent difference in behaviour.

  System:
    OS: Windows 8.1
    CPU: x64 Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
  Binaries:
    Node: 8.10.0
    Yarn: 1.3.2
    npm: 5.6.0
  npmPackages:
    jest:
      wanted: ^22.4.2
      installed: 22.4.2

[thymikee]

Jest runs tests in parallel when it "makes sense" – which is determined by some heuristics, like number of tests or how long do they execute. Sometimes it just makes sense to run your tests in one worker (in band) because spawning multiple workers takes time.

[gsteacy]

It does seem to correlate with test time now that you mention it. Some of our test suites frequently but inconsistently take upwards of 5 seconds (which seems to be due to filesystem thrashing as there are only a few tests), but evidently that impacts the heuristic. Thanks for your response.

[SimenB]

If interested, the algorithm is here:
https://github.com/facebook/jest/blob/1a487c1803124c594bdd86ee24b5949025660bc3/packages/jest-cli/src/test_scheduler.js#L33
https://github.com/facebook/jest/blob/1a487c1803124c594bdd86ee24b5949025660bc3/packages/jest-cli/src/test_scheduler.js#L88-L96

[dan-kez]

@thymikee Is there a way to force running in parallel in CI? I'm running into an issue where my tests are very slow in CI since they appear to be forced to run in band.

[thymikee]

Create at least 21 test files and invoke with --maxWorkers=4 to run tests on 4 threads for example. That should be enough to trick Jest into running your tests always in parallel.

[dan-kez]

Thanks for the fast reply @thymikee!

It's unfortunate that there isn't a way to force it to run with less than 21 test files. In our case we have 4 files that run chromeless lambda for visual regression testing. Each test takes ~ 10 sec. I can make a number of "empty" files to force this behavior but that's a little messy.

Is there any desire to add a flag that ignores the logic @SimenB linked to above?

[gsteacy]

As I mentioned in the issue description, if you run with --no-cache it always seems to run in parallel (as long as there are at least two files anyway). I haven't looked into exactly _what_ jest caches, but based on the algorithm above and my anecdotal evidence it seems that this flag throws away prior run stats, so Jest has no timings to decide if it should run in band. Of course the cache is there for a reason, but given that your tests already take 10 seconds it might make any performance gains from the caching moot anyway. Only one way to find out!

[cyphercodes]

6183 feature request for max concurrent in a single test file which can be a solution to this issue as well.

[gae123]

Not sure whether this is a very recent regression somewhere in the stack, but I just discovered that around upgrading to 24.8.0, I had to remove--detectOpenHandles if I wanted tests to run in parallel. This was not the case until very recently.