Jest: Issue with jest 22.4.2 and 0 exit code on failure

[manu-st]

Since our upgrade to 22.4.x, our batch scripts are not detecting when tests are failing because it always returns 0 as exit code. We were previously using 21.2.1 which worked properly.

I'm not completely sure, but it seems to be related to the changes made in #5313 and #5332. Any idea why it is not set properly anymore?

This is on Windows.

[SimenB]

Can you set up a reproduction repo?

/cc @mjesun

[provAlexSage]

Here's some extra info for whoever investigates this.
I stopped getting this issue after downgrading from 22.4.3 to 22.4.1

[mjesun]

We definitely need something to repro. Both changes mentioned were done way before 22.4.1 so it needs to be something else. Getting a confirmation that it only happens on Windows would also help.

[mcblum]

@SimenB we're also seeing the same issue on 22.4.3. We've started with this version so it's possible that a downgrade would fix it. I'm not seeing it locally on my Mac, but I am seeing it on our build server (Ubuntu 16.04).

[manu-st]

I don't have a reproducible demo yet, however I found out that it only occurs if I perform code coverage as part of testing. If I do not do the code coverage, then the error is properly reported back.

[manu-st]

Actually, the coverage has nothing to do with that problem, it is just that I was also using --bail when not performing the coverage. And when --bail is specified it does a hard exit.

In my case, I managed to fix using various ways:

• use --forceExit which causes a hard exit with the proper error code
• modify readResultsAndExit to call process.exit in the exit handler
• update my code to read the value of process.exitCode and if non-zero, I would call exit.

Because I've been using electron and not node to run jest and that I run jest inside the renderer process of electron the process.exitCode set by jest had no effect. In the old version, in the readResultsAndExit handler it was calling exit and thus it worked in both node and electron. Now setting process.exitCode has no effect for electron.

For those interested, here is the electron.js file I'm using to run jest inside electron that properly detect the exit code of jest:

const { app } = require ("electron");
const jest = require("../node_modules/jest-cli/build/cli");

app.on ("ready", async () =>
{
   try
   {
      await jest.run ();
      app.quit ();
      if (process.exitCode !== 0)
      {
         app.exit (process.exitCode)
      }
   }
   catch (e)
   {
      // Exit with an error code when test are failing
      app.exit (1);
   };
});

[paulbrittain]

This is still an issue with the latest version of Jest, and a few (if not all) versions in the past 8 months. On Windows.

[AndreMiras]

Yes pretty annoying, I also have the same issue on Ubuntu 18.04.
The good thing is it behave as expected with the CI environment, see demo below:

yarn test --watchAll=false &> /dev/null
echo $?
0
CI=1 yarn test --watchAll=false &> /dev/null
echo $?
1

[mAAdhaTTah]

I can confirm @AndreMiras' behavior with [email protected].

[jchannon]

Still see this issue in 24.8.0 but can't get @AndreMiras approach to work. The exit code is always 0 on a failure

[Srinath-Ganesan]

I also still see this issue in 24.8.0 and 24.9.0. can't get @AndreMiras 's approach to work :( . The exit code is always 0 on any failure.