Ava: Customize assertions

[yamsellem]

Is there a way to add some custom assertions to t existing ones?

I would like to add those basic ones:
— t.instanceof(object, clazz)
— t.length(array, length)
— t.members(array, item, item)
— t.includes(string, substring)

I guess t.context can be used, but feels cumbersome.
And maybe, those assertions can be considered to be added to ava core.

[sindresorhus]

Not at the moment, but your needs can be solved with the existing methods:

t.true(object instanceof clazz);
t.is(array.length, length);
t.true(array.every(x => [item, item].includes(x)));
t.true(string.includes(substring));

You can already use any external assertion library. The only caveat is that t.plan() doesn't work.

[alathon]

I think a very strong point of AVA (and tape) is precisely that there are few assertions by default - this reduces the amount of 'magic' that occurs, and reduces bugs in testing code that occurs because you aren't 100% sure how a particular assertion works. Given that bugs occur almost as frequently in test code as they do in the code they test, I think it would be a mistake to change this.

[yamsellem]

@alathon Being small does not necessarily means being ascetic.

t.is(array.length, length); // is better with t.truthy(array) before
t.true(array.every(x => [item, item].includes(x))); // seems cumbersome
t.true(string.includes(substring)); // could be simpler

Some basic assertion on strings and arrays seems fair to me — and don't break the 'no magic' of the thing, IMO.

[sotojuan]

I guess the problem is that it's a slippery slope. These kind of extra assertions are better in userland as separate modules, functions, or snippets. I personally don't see anything wrong with the code block in the last comment but you could make it look "prettier" by abstracting those functions away. That way you can use your own custom logic as well.

The smaller core AVA is, the easier it is to use.

[vadimdemedes]

@sotojuan Good point, totally agree.

I think we should make t extendable with custom methods (assertions). Was thinking to create a separate module for t.jsxEqual(), would be nice to make this possible.

[odigity]

I'm in the same boat. I would like to write some custom assertions for convenience and add them to t in my project.

The docs say "you can use custom assertions", but provide no information on how.

[vadimdemedes]

@odigity section "Custom Assertions" in readme is actually related to using 3rd-party assertion libraries, like expect or chai, not extending t with custom assertions. I understand the confusion, I think we have to rename the title to something like "Using 3rd-party assertion libraries".

[odigity]

So... what's the easiest way to extend t with custom assertions? :)

[novemberborn]

@odigity There is no way to do this at the moment. To start we'd need a proposal on a) extending AVA, b) how to register new assertions, and c) how this would integrate with power-assert and our assertion output.

[odigity]

So how do you recommend approaching this particular use case?

I need to compare two floats with some margin of error. Because there's no support built-in, and no documented way to add custom assertions, I'm currently doing this:

t.is( Math.round(actual_duration), Math.round(expected_duration) )

(It's a video processing engine.)

It works, but it's ugly compared to, say...

t.almostEquals( actual_duration, expected_duration )

Perhaps with an options arg to control the error margin.

_almostEquals_ is a common term used when dealing with floats in JavaScript.

• Exhibit A (Yes, I'm thinking about pulling this in to my project.)

Note: I'm not asking you to implement this, only if there exists a sane way by which one such as myself could do so.

[novemberborn]

@odigity I'd do t.true(almostEquals(actual, expected)).

[alathon]

@novemberborn But then you don't get the values for actual and expected, just a true or false value. That makes it (potentially) harder to debug. That depends on what you're testing, though, I suppose -- are you testing an almostEquals function, or that actual and expected are close enough; if its the function, then yes that'd be the way to go.

I don't see what's wrong with t.is( Math.round(actual_duration), Math.round(expected_duration) ) though, if thats what you're using (Math.round). That seems clear and declarative to me. Anyone looking at it will know exactly what's going on - as opposed to t.almostEquals, which beckons the questions of: Almost equals according to which precision/function? Can it be used on objects/strings? etc..

[novemberborn]

But then you don't get the values for actual and expected, just a true or false value.

For t.true() we also print the values of the expressions in the argument, so you should see those values.

[alathon]

Ah, my bad -- was thinking of tape when it comes to that.

[epeterson320]

+1 to being able to extend t since we users can get the features we want and the core API can remain small. Jest has a nice API to extend their matchers. Perhaps a similar method would work in AVA.

For what it's worth, though, I'd like to see an almostEquals, closeTo, or inDelta matcher in core. Missing instanceOf is easy enough to work around, but comparing floating point numbers with a certain precision gets either verbose or hacky when testing compute-focused code.

[lewisdiamond]

It's really worth making it easy to provide an implementation of custom tests. Right now using t.true(_.isMatch(... ...)) outputs almost pure gibberish. It should only print the difference so it's really easy to spot.

[novemberborn]

@lewisdiamond would you want to help us implement this?

[chocolateboy]

In the meantime, the following hack works for me currently (AVA 2.1.0, 3.8):

import test from 'ava'
import { Assertions } from 'ava/lib/assert.js'

Object.assign(Assertions.prototype, {
    isFoo (value) { this.assert(value === 'foo') },
    isBar (value) { this.is(value, 'bar') }
})

test('foobar', t => {
    t.isFoo('foo')
    t.isBar('bar')
})

The only nits I've spotted are that custom assertions don't support the skip modifier, and the line number for assertion failures points to the assertion call inside the custom assertion rather than its caller (the offending line can be gleaned from the stack trace).

[gajus]

I've used the latter approach to implement https://github.com/gajus/ava-dom

As already mentioned by @chocolateboy, the biggest problem is that the error points to the location of t.true not the assertion.

Is there a way to workaround this limitation?

[novemberborn]

If we could somehow expose a way to report an assertion error I wouldn't be opposed to that. Maybe as a symbol property on t. And then if other stuff is a bit hacky or just a helper method you pass t to then maybe that's enough for now?

2435 suggests an API to construct custom test() functions, with full TypeScript support, and that could be a way to register additional assertions I think. But we haven't worked out at all how that would be forward compatible with other changes AVA core would like to make to the t object or how to best distribute those implementations.