Jest: Plan Total Assertion Count

[browniefed]

Currently looking at adding the ability to plan the amount of assertions, which will be useful for async testing.
Currently I see there is access to result.failedExpectations.length + result.passedExpectations.length in the specResultCallback. So getting the total is easy.

However struggling to figure out the best way to set up the expectation count w/ expect call. Like expect().toHaveAssertionCount(5) (terrible name).

Any pointers on a direction to go on this would be great.
Like catching a known error in Spec.prototype.onException?

[bookman25]

What about if the code could instead analyze how many expect calls are defined in the block and verify that they are all hit. So if you had 5 expect's and only 4 got hit, it would error letting you know some async code never executed.

[browniefed]

@bookman25 are you talking about statically analyzing the amount of calls to expect?

[bookman25]

As costly as it may be, yes. That's exactly what I was talking about. Not saying it's the best idea, but we have a lot of cases where our expect's weren't getting hit. And in some cases the tests should have failed but passed because the expects were never triggered.

[browniefed]

That's not necessarily doable or reliable. Changing the count of assertions could theoretically be dynamic and not statically analyzable.

[bookman25]

Yeah, I'm aware of that. It's more of a pipe dream. It could handle cases where the number of expects are static, but not all cases. And as a result would probably lead to buggy tests for the dynamic cases. :(

[cpojer]

I think we shouldn't do this statically. It should be a runtime check after the test completes.

[aaronabramov]

het @browniefed
per our discussion today:
here's the hook that i used to fail the test if there's a failing .toMatchSnapshot (they don't throw by default: https://github.com/facebook/jest/blob/master/packages/jest-jasmine2/src/setup-jest-globals.js#L22-L34

and you can use those to set this global matcher state from the inside of expect code: https://github.com/facebook/jest/blob/master/packages/jest-matchers/src/index.js#L144-L148

[browniefed]

Well that was fun, things learned. Run npm run build before npm run jest or you'll be sad.
https://github.com/facebook/jest/pull/2133

[piuccio]

Really interested in this, I wonder why so few test runners have implemented it.

Very often I tests like the one below when testing error conditions

describe('what is dead', () => {
    it('may never die', () => {
        return drown(euron)
        .catch(() => {
            expect(euron.isAlive()).toBe(true);
        });
    });
});

However if the drown() returns a fulfilled promise, the test will pass, which is wrong.

So I have to write something like

return promise.then(() => throw new Error('for the watch')).catch(...)

But being able to say how many assertion you're planning to run is much shorter

it('respects the plan', () => {
   plan(1);
   // ...
});

http://www.node-tap.org/api/#tplannumber

[cpojer]

This will be in the next Jest release as expect.assertions(number).

[aymericbeaumet]

@cpojer Any news on this? Thanks!

[browniefed]

This is already released @aymericbeaumet

[aymericbeaumet]

Oh great! Didn't find it in the docs