Jest: Custom error messages for assertions

[brigand]

I'm guessing this has already been brought up, but I'm having trouble finding the issue.

Do you want to request a feature or report a bug?

Feature.

What is the current behavior?

expect(false).toBe(true, "it's true") doesn't print "it's true" in the console output.

If the current behavior is a bug, please provide the steps to reproduce and either a repl.it demo through https://repl.it/languages/jest or a minimal repository on GitHub that we can yarn install and yarn test.

What is the expected behavior?

The message should be included in the response somehow.

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.

Just made a new project

yarn add --dev jest
echo 'it("works", () => { expect(true).toBe(false, "FOO"); });' > a.test.js
./node_modules/.bin/jest

[cpojer]

There are many questions here, one of them in this issue #1965.

[disklosr]

Why was this closed? The linked discussion doesn't mention custom error messages! Is this supported in jest?

[phawxby]

@cpojer is there a way to produce custom error messages?

I'm using lighthouse and puppeteer to perform an automated accessibility audit. I don't know beforehand how many audits are going to be performed and lighthouse is asynchronous so I can't just wrap each audit result in the response in a test block to get a useful error message.

I'm left with

        await page.goto(url);

        const result = await lighthouse(url, { port: port }, lighthouseConfig);

        const auditResults = Object.values(result.audits);

        for (let result of auditResults)
        {
          expect(result.score).toEqual(true);

        }

Which then returns

    Expected value to equal:
      true
    Received:
      false

So if I have a single audit failure I just get expected whatever to be true, it was false but with no information as to which audit failed.

This is the only way I could think of to get some useful output but it's not very pretty.

        for (let result of auditResults)
        {
          let output = true;

          if (!result.score) {
            output = `${result.name} - ${result.displayValue}`;
          }

          expect(output).toEqual(true);
        }

[SimenB]

@phawxby In your case I think a custom matcher makes the most sense: http://facebook.github.io/jest/docs/en/expect.html#expectextendmatchers

Then you can use jest-matcher-utils to create as nice of a message that you want 🙂 See https://github.com/jest-community/jest-extended/tree/master/src/matchers for a bunch of examples of custom matchers

If you do create the custom matcher(s), it would be awesome to link to them in http://facebook.github.io/jest/docs/en/puppeteer.html

[phawxby]

@SimenB that worked really well. The whole puppeteer environment element was overkill for my needs as not all the tests require it but here's what I used.

const puppeteer = require('puppeteer');
const lighthouse = require('lighthouse');
const url = require('url');
const { getSlugs } = require('./bulk.js');
const config = require('./lighthouse-config.js')

expect.extend({
  toPassLighthouse(received) {

    const auditResults = Object.values(received.audits)
    .filter((x) => !x.manual); // Strip manual audits. We don't care about those inside automated testing ;)

    for (let result of auditResults)
    {
      let output = true;

      if (!result.score) {
        output = `${result.name} - ${result.description}`;

        if (result.displayValue) {
          output += ` - ${result.displayValue}`;
        }

        return {
          message: () => output,
          pass: false
        };
      }
    }

    return {
      message: () => `expected to fail lighthouse`,
      pass: true
    }
  },
});

let browser;
let page;
let port = null;

beforeAll(async () => {
  browser = await puppeteer.launch();
  page = await browser.newPage();

  const endpoint = new URL(browser.wsEndpoint());
  port = endpoint.port;
});

afterAll(async () => {
  await page.close();
  await browser.close();
});

describe(`Accessibility`, () => {
  const slugs = getSlugs();

  for(let slug of slugs)
  {
    let url = `http://localhost:3000/patterns/${slug}/${slug}.rendered.html`;

    test(slug, async () => {
      await page.goto(url);

      const result = await lighthouse(url, { port: port }, config);

      expect(result).toPassLighthouse();
    });
  }
});

[shaunc]

sigh... ok: so its possible to include custom error messages. Still (migrating from mocha), it does seem quite inconvenient not to be able to pass a string in as a prefix or suffix. I would think this would cover many common use cases -- in particular expect() in loops or in a subroutine that is called more than once.

UPDATE

Ok .. not to undercut the case, but a workaround is changing expect(result).toEqual(expected) to:

const msg = '... my message...'
expect({msg, result}).toEqual({msg, result: expected})

[sharikovvladislav]

@cpojer @SimenB

So any approaches how to provide a custom message for "expect"?

[SimenB]

Use assert instead of expect is the current workaround if you really need it

[sharikovvladislav]

@SimenB ok ty! it worked.

[mmkal]

@cpojer @SimenB I get that it's not possible to add a message as a last param for every assertion. But what about very simple ones, like toBe and toEqual? I think that would cover 99% of the people who want this. toHaveProperty will already give very readable error messages. When I use toBe and toEqual it's usually because I have some custom condition that jest can't easily help me assert on out-of-the-box. It's especially bad when it's something like expected "true", got "false".

Can we reduce the scope of this request to only toBe and toEqual, and from there consider (or not consider) other assertion types?

[dogboydog]

toBe and toEqual would be good enough for me. Personally I really miss the ability to specify a custom message from other packages like chai. expected 0 to equal 1 usually means I have to dig into the test code to see what the problem was. I would appreciate this feature

[jcollum]

I just use chai.should:

__setup.js:

global.should = require('chai').should();

in a test:

result.should.have.property('URL', 'abc', 'result.URL did not have correct value');

When things like that fail the message looks like: AssertionError: result.URL did not have correct value: expected { URL: 'abc' } to have property 'URL' of 'adbc', but got 'abc'

Which, to me, is much nicer.

[mattphillips]

Posting this here incase anyone stumbles across this issue :smile:

jest-expect-message allows custom error messages for assertions.

test('returns 2 when adding 1 and 1', () => {
  expect(1 + 1, 'Woah this should be 2!').toBe(3);
});

/*
  ● returns 2 when adding 1 and 1

    Custom message:
      Woah this should be 2!

    expect(received).toBe(expected) // Object.is equality

    Expected: 3
    Received: 2
*/

[mrded]

You can also throw an error following way, without using expect():

it('foo should be true', (done) => {
  if (foo === true) {
    done(); // All good.
  }
  else {
    done(new Error('Sorry, it is false'));
  }
});

It comes handy if you have to deal with a real async code, like bellow:

it('promise should resolve, (done) => {
  fooPromise()
    .then(() => done())
    .catch(done);
});

[SimenB]

When you have promises, it's highly recommended to return them.

it('promise should resolve', () => {
  return fooPromise();
});

// or

it('promise should resolve', async () => {
  await fooPromise();
});

That will behave the same as your example

[amis47]

Posting this here incase anyone stumbles across this issue 😄

jest-expect-message allows custom error messages for assertions.

test('returns 2 when adding 1 and 1', () => {
  expect(1 + 1, 'Woah this should be 2!').toBe(3);
});

/*
  ● returns 2 when adding 1 and 1

    Custom message:
      Woah this should be 2!

    expect(received).toBe(expected) // Object.is equality

    Expected: 3
    Received: 2
*/

fwiw: it works well if you don't use flow for type checking

[Asday]

All of the above solutions seem reasonably complex for the issue. besides rolling the message into an array to match with toEqual, which creates (in my opinion) ugly output.

In that spirit, though, I've gone with the simple:

const duplicateErrors = max(values(countBy(values(errors)))) > 1
// Add some useful information if we're failing
if (duplicateErrors) {
  console.log('actual errors\n', errors)  // eslint-disable-line no-console
}
expect(duplicateErrors).toBe(false)

Jest's formatting of console.log()s looks reasonably nice, so I can easily give extra context to the programmer when they've caused a test to fail in a readable manner. Logging plain objects also creates copy-pasteable output should they have node open and ready.

[gnapse]

Use assert instead of expect is the current workaround if you really need it

@SimenB perhaps is obvious, but not for me: where does this suggested assert come from? I search for it in jestjs.io and it does not seem to be a jest api.

[Asday]

Looks like a node thing.

> assert(1 === 2)
Thrown:
{ [AssertionError [ERR_ASSERTION]: false == true]
  generatedMessage: true,
  name: 'AssertionError [ERR_ASSERTION]',
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '==' }
> 

[jasperkuperus]

Thanks @mattphillips, your jest-expect-message package works for me! Especially when you have expectations in loops, this functionality is really important.

[roschaefer]

For those of you who don't want to install a package, here is another solution with try/catch:

Code

How it looks

Pull Request for Context
https://github.com/Human-Connection/Human-Connection/pull/1553

Code to copy+paste

        it('fails with a custom error message', async (done) => {
          try {
            await expect(somePromise()).resolves.toMatchObject({foo: 'bar' })
            done()
          } catch(error) {
            throw new Error(`
              ${error}
              Write a helpful error message here.
            `)
          }
        })

I want to show a custom error message only on rare occasions, that's why I don't want to install a package. In our case it's a helpful error message for dummies new contributors.