Jest: Run each test case in a separate process / better isolated context?

[martijnthe]

Do you want to request a _feature_ or report a _bug_?

Request a feature.

What is the current behavior?

Currently, tests in a single test source file are run one after the other in the same process / context.
One can use resetModules and resetMocks (disabled by default!?) to reduce the chances of one test to affect another test.

One of the problems with resetModules and resetMocks is that this doesn't jive very well with having ES6' top-level import declarations in your test source file. They create references to the initial module / mock instances that resetModules and resetMocks won't be able to "reset" to new instances. Someone had already filed a ticket for that in the past: https://github.com/facebook/jest/issues/3236

Instead of trying to reset the state after each test case (which is hard!), I think it's better to run each test case in "true isolation" in a separate node.js process (or perhaps a lighter way of isolating, I'm not sure what it would be though).

Please provide your exact Jest configuration and mention your Jest, node,
yarn/npm version and operating system.

Using:

• Jest v22.2.2
• Node v8.9.1
• Yarn v1.3.2

[SimenB]

How would you run it in "true isolation" if you want to use static import? Those are mutually exclusive

[martijnthe]

How would you run it in "true isolation" if you want to use static import? Those are mutually exclusive

@SimenB not sure if I follow. Why are they mutually exclusive? I imagine the thing that orchestrates running test cases to spin up a process, passing the name of the (single) test case to run. The new process would then only run the test case matching the name of the test that was passed in by the orchestrator.

[martijnthe]

How would you run it in "true isolation" if you want to use static import? Those are mutually exclusive

Just talking with @hbehrens who pointed out that you can already use static imports and run in "true isolation" by running jest "manually" for each test case, using --testNamePattern to make sure only single test case gets run. It would of course be terribly slow and cumbersome, but just to prove the point that static imports and "true isolation" isn't mutually exclusive.

[cpojer]

Right now we are not planning on adding such a feature mainly because of how much the added splitting or static analysis would add to Jest's runtime. Basically, we'd have to either run the startup code for each file in each process or do static analysis to extract the relevant code to run in isolation. Jest provides complete isolation between files, manual isolation within files, like you pointed out.

If you'd like to do this regardless, there are two options. One quick one, and one slow one:

• Split up your tests into individual files.
• Watch https://www.youtube.com/watch?v=NtjyeojAOBs and build your own test runner for Jest that does this splitting. If you build it, and it is awesome, we may even want to include it in Jest some day!

[HBehrens]

Thank you for the hints, @cpojer – @martijnthe also had the idea of relying on dynamic import to "re-require" each symbol on each test to take advantage of resetModules.
To avoid manual boiler plate and keep TS type-safety, one could use a Jest transformer. An import like

import {foo} from 'bar';

could be transformed to

let foo; env.beforeEach(async () => {{foo} = await import('bar');});

This is still not 100% isolation as there could be shared state within a single test file and further introduces a problem where one cannot use these symbols outside of a test (may be identified at runtime by (re-)initializing each symbol with a values such as 'only accessible during a test'). But it would remove the need to write a custom runner + code splitting.

We will look into this and report back here.

[borela]

. Jest provides complete isolation between files

@cpojer Does that mean that each test suite(file) run on its own process? By the way I loved the platform concept you guys are implementing on jest.

[samfeder]

as @cpojer recommended, simply splitting my test into two files resolved this issue for me. One might even argue it's cleaner to split into two files if you need to re-initialize a require.

[hugollm]

I actually needed this for some tests with puppeteer. I'm probably better off refactoring the setup or test logic, but meanwhile I got isolation with:

jest --listTests | xargs -L 1 -P 0 jest

This runs each file as a separate jest command, so each of my test files spawn a new puppeteer context, solving my problem for now. Figured I would post this here in case it helps someone.

[sseletskyy-apixio]

jest has a built in flat to run all tests sequentially

jest --runInBand

alias -i

I used that for running just test against storybook+puppeteer environment (e.g. npm lib spdt)

[glasser]

@hugollm I'm not sure that's what's requested: it still runs all the tests in a file the same process.

I'm interested in this not as an ordinary way to run tests all the time, but to occasionally confirm that there aren't any accidental inter-test dependencies that will cause it.only and friends to fail. Doesn't look like there's a good way, though!

[silverwind]

I'd also have a use case for this for stdin testing. I write data to process.stdin like this and emit a end event on it. The problem is that it can only be done once per process because stdin cannot be reopened. Ideally, I'd like every test call to get it's own process with a fresh stdin.

[SimenB]

@silverwind yeah, that's a valid use case. I've been wanting to give tests fresh/separate process.{stdout,stderr,stdin} for some time - they are currently the real ones (we swap out exit, but other stuff like chdir is also the real ones). Wanna open up an issue so we can track it?

[silverwind]

Sure, https://github.com/facebook/jest/issues/10039