Jest: Bug: --bail flag does not bail

[JoshCheek]

Do you want to request a feature or report a bug?

Report a bug.

What is the current behavior?

The --bail flag does not stop running tests after the first failure.

Steps to reproduce

$ git clone https://github.com/JoshCheek/jest-bug--bail-flag
$ cd jest-bug--bail-flag/
$ npm install
$ npm test

What is the expected behavior?

Exit the test suite immediately upon the first failing test, the one named "2: fail".

What is the actual behaviour?

It runs the full test suite.

Demo

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.

No configuration outside the repo I linked above..

$ npm run exec jest -- --version
v18.1.0

$ node --version
v7.2.1

$ npm --version
3.10.9

$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.11.6 (15G1004)
      Kernel Version: Darwin 15.6.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: Josh’s MacBook Air (2)
      User Name: Josh Cheek (josh)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 19 days 12:03

[cpojer]

--bail in Jest works across test suites, not individual tests. If you'd like to focus on a specific test that you are fixing, use -t or fit/it.only.

[JoshCheek]

That's not my use case, I need it to exit the test suite immediately upon the first failing test. This is what Mocha's --bail does (presumably where Jest got it from), it's what RSpec's --fail-fast does (presumably where Mocha got it from).

Seems like Jest should either change the behaviour to match the docs or change the docs to match the behaviour. My experiences lead me to advocate the former as it is the specification that the implementation must satisfy.

[cpojer]

Please feel free to change Jest/our fork of Jasmine to implement this feature but this is never how bail worked for Jest. I'd be open to merge a PR if you'd like to change behavior here.

[JoshCheek]

I don't think the link made it into the message, can you post again?

[cpojer]

I didn't add a link. This change would have to be made somewhere in jest-jasmine2 or jasmine itself.

[tonyxiao]

@JoshCheek did you end up implementing a jasmine extension to fail fast?

[tonyxiao]

Actually I was able to get this to work via my setupTestFramework.ts file.

import * as failFast from 'jasmine-fail-fast'
/** 
 * Fail after the first test in a single test suite fails. This is NOT the same as jest's
 * --bail option, which works across test suites
 */
if (process.env.FAIL_FAST) {
  // Jasmine definition from @types/jest does not have `getEnv` for some reason
  // https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/jest/index.d.ts
  // However this empirically works with jest 22.3.0. So we'll cast to any for now
  const jasmineEnv = (jasmine as any).getEnv()
  jasmineEnv.addReporter(failFast.init())
}

[Akuukis]

Thanks to tonyxiao, here's how I managed to "fix" --bail behavior. (Jest 24.5.0)

// setup.ts
import * as failFast from 'jasmine-fail-fast'

if (JSON.parse(process.env.npm_config_argv).original.includes('--bail')) {
  const jasmineEnv = (jasmine as any).getEnv()
  jasmineEnv.addReporter(failFast.init())
}

[krzkaczor]

Adding my two cents 🤑:

1. In your jest.config.js add:

  setupFilesAfterEnv: ["./setup-after-env.js"],

1. Install yarn add jasmine-fail-fast

2. setup-after-env.js:

const failFast = require("jasmine-fail-fast");

if (process.argv.includes("--bail")) {
  const jasmineEnv = jasmine.getEnv();
  jasmineEnv.addReporter(failFast.init());
}

Works like a charm! 🎉

[A-Babin]

The solution above works, but when running jest with a glob, setup-after-env is run in a child process. The --bail flag is not passed to it, failFast won't be enabled, and all tests will run.

I think executing with --runInBand solves that problem, but you lose parallelism.

Instead, I've wrapped the failFast setup in a utility function and call it where it's needed. On a failure or error inside a file where it's executed, all tests inside that file will be skipped, but not the entire test run.