Jest: Feature Request: Allow to run a jest runner in band

[DanielMSchmidt]

Do you want to request a _feature_ or report a _bug_?
Feature

What is the current behavior?
When I have a third-party testing tool that is not allowed to run in parallel I am unable to use it in the integration with jest, besides running every single test in band. The concrete thrid-party tool is cypress, it can't be run in parallel.

In my setup I use jest, jest-runner-eslint, jest-runner-stylelint (jest-runner-tsc is planned) and I would like to add something like jest-runner-cypress. As I can not share a lock between two or more processes (without using redis or something similar) I am unable to build / extend such a runner to run the processes in band. As far as I can see there are two possible solutions to this problem:

1) User Land
Allow a project to be configured as runInBand, reverting the decision already made by @cpojer

2) Runner Land
Let the runner expose a config that sets it as runInBand only

This relates to https://github.com/facebook/jest/issues/3215 but I wanted to describe my use case a bit more in detail as well as the different solutions that would be possible for my problem, I hope this was okay.

Thank you for taking the time to read through this 👍 If you make a decision I would be happy to provide a PR (or at least to try to do so)

[cpojer]

Thanks for bringing this up. Definitely an interesting use case. I think I'd prefer this to be triggered by an individual runner and be scoped to that runner specifically rather than making the change for users. We can then make it so that one runner runs in band always, while the others run in parallel, unless --runInBand is passed to the whole run. Do you wanna work on a PR that makes this possible for Jest runners?

[DanielMSchmidt]

I would love to give it a try, can I ping you in the PR for early feedback once I am this far?

[cpojer]

Absolutely!

[DanielMSchmidt]

Do I get it right that I would need to change this part to be something like this

await testRunners[runner].runTests(
   testsByRunner[runner],
   watcher,
   onStart,
   onResult,
   onFailure,
   {
        serial: runInBand || runner.serialOnly,
   },
);

If that's it I am actually super impressed by the internal API design that it allows me to specify this so easily 😲

[cpojer]

Yeah, that's it. I'm curious to see what happens to performance when you make that change.

[ElliotChong]

Wanted to chime in and mention that I have a tangential use-case that involves projects / user-land configs:
I have typical Jest unit tests which should be ran in parallel using the default Jest behavior and end-to-end Puppeteer-based tests which ideally run in serial with runInBand.

Currently I execute these test suites as separate Jest commands to achieve this, but being able to define runInBand at a config level would be useful as I don't use a separate runner to execute Puppeteer tests. FYI- Puppeteer setup is pulled more or less straight from the official docs.

[lgandecki]

Btw @DanielMSchmidt I worked on something like this a while ago, https://github.com/TheBrainFamily/jest-runner-cypress-example / https://github.com/TheBrainFamily/jest-runner-cypress

It would be great to be able to run those inBand, and the rest in parallel :)

[SimenB]

See #5712 for open PR

[DanielMSchmidt]

@lgandecki This is the exact reason I am doing this PR, at dcos-ui we have eslint, stylelint, jest and cypress and I would love to see them all united as runners under jest

[lgandecki]

@DanielMSchmidt do you already have a solution for a cypress jest runner? Or would you like to collaborate on something? This is a topic of a high interest for me as well :) Especially with the new features of cypress that let you mount react components directly through cypress, to do more of a "unit" kind of testing - the runner would be great to have in that case (since we import the file a component, chaing the file would give the indication to the jest runner to rerun the proper cypress test)

[DanielMSchmidt]

I worked on one to try it out myself and got basically as far as you, except that I use the binary. I would love to collaborate on this, let's make it a shared effort.

[lgandecki]

@DanielMSchmidt I just saw your repo with the runner - cool! And thanks so much for giving credits to our work here at TheBrainFamily :) I cried a little when I saw it ;-)

We should probably move out of here to not pollute jest issues page with our discussion. One question that comes to my mind is - what do you think is an advantage of using the binary instead of the api? what I would love to do is to be able to use previously opened cypress window. The test start time would decrease greatly.. and we would have a way to visually debug the tests - which I think is one of the main selling points of cypress.

[DanielMSchmidt]

@lgandecki I opened an issue on jest-runner-cypress to move this discussion to 👍