Jest: `expect`s should fail a test even if it's inside a try/catch.

[sebmarkbage]

Currently expect helpers throw if there's an error. This lets them be caught in various global error handlers. This makes it difficult to test environment that has expected thrown behavior or explicitly tests custom error handling behavior.

Failing assertions should set a global flag that fails the test even if there's a try/catch involved.

For example, these tests should fail:

  it('should fail this', () => {
    function frameworkCode(callback) {
      try {
        callback();
      } catch (x) {
        console.log('An error was handled by the framework.');
      }
    }
    function callback() {
      expect(false).toBe(true);
    }
    frameworkCode(callback);
  });

  it('should fail this too', () => {
    function callback() {
      expect(false).toBe(true);
      throw new Error();
    }
    expect(callback).toThrow();
  });

[sebmarkbage]

For real world use cases see React repo that tests lots of behavior that is expected to throw but is also expected to issue console warnings, or not issue console warnings.

[sebmarkbage]

The current behavior also cover up error like this one:

  it('should fail this too', () => {
    function returnTrue() {
      return false;
    }
    function callback() {
      expect(returnTrue()).toBe(true);
      if (false) {
        throw new Error();
      }
    }
    expect(callback).toThrow();
  });

There are two errors here, but they cancel out each other so it looks like it passes but it should fail.

[sebmarkbage]

Another case of bugs being covered up by this https://github.com/facebook/react/pull/10331

[aaronabramov]

we already have a state where keep all the information about our expects for each test.
It should be really easy to implement.

should we make it configureable, or should we always fail the test if any of the expect's throw?

[sebmarkbage]

It might be nice to make it configurable as an upgrade path but I don't see why you'd expect anything else.

[cpojer]

I agree this needs fixing, I don't agree this needs a configuration. I'm fine with the breaking change here, it exposes bugs.

[marxlow]

I am facing the same problem with jest that expects are not working as they should inside try-catch blocks.

it('I should be failing', () => {
  try {
    IWillNotThrowAnyError();

    // Code should reach here
    expect(false).toEqual(true);
  } catch(error) {

   // Code will never reach here
    console.log('Caught an error!');
    expect(true).toEqual(true);
  }
}

The jest test above oddly passes.

[sebmarkbage]

cc @bvaughn

[cfnelson]

Experiencing same behaviour on Jest: 22.0.5, whilst using expect within a try/catch block.

Below is a minimal working example of the unexpected behaviour (without). The expect is called as expect.assertions(1); passes without issue, console.error also reports the matcherResult which indicates it should have failed.

test("It should FAIL, it reports PASS and logs error to console.",  () => {
  expect.assertions(1);
  try {
    expect(false).toEqual(true);
  } catch (error) {
    console.error('😭😭😭', error);
  }
});

// Expect reports the below matcherResult
// matcherResult: { 
//  actual: false,
//  expected: true,
//  message: [Function],
//  name: 'toEqual',
//  pass: false } }

Interestingly expect.assertions() behaves differently to expect() and will FAIL the test whilst it's within a try/catch block. See below example & output.

test("It should FAIL, it expects two assertions and it FAILS for receiving only one assertion.",  () => {
  try {
    expect.assertions(2);
    expect(true).toEqual(true);
  } catch (error) {
    console.error('😭😭😭', error);
  }
});

Hopefully this will be helpful to someone. 😄

[cvle]

We also ran into this. As a workaround we added a separate global function expectAndFail that wraps the original expect but fails the test immediately when the assertion fails.

https://gist.github.com/cvle/e4496b733889c137e9f19dddf861fdd4

You could also replace global.expect with this if you really wanted, but it's a breaking change for some libraries that depends on the non failing behavior.

[esamattis]

That's clever, thanks for sharing!