Jest: Feature request: toMatchFileSnapshot

[gre]

When testing a graphics library (e.g. canvas, webgl), you want to test what you draw against a reference image.
toMatchSnapshot is great, but it's meant for small objects, code, (so you can actually look in the file, or look at the diff, and see if it is correct). toMatchSnapshot don't work that great for images or any binary content. You actually expect a binary content is kept verbatim and saved in a single snapshot file so you can see it and verify it matches your expectation.

Proposal

---

expect(blob | buffer | string).toMatchFileSnapshot();

Other thoughts

---

complementary we could imagine this as well:

expect(blob | buffer | string).toMatchFile("./foo.png");

Also, I'm wondering if it would be easy to support a Stream as well?

expect(binaryStream).toMatchFileSnapshot();

[cpojer]

Hey,

this definitely sounds like a very interesting thing to add and I'd be happy to support this in Jest. I don't think we'll be able to spend time on this ourselves at the moment but is this something you are interested in building?

How do you imagine we'd be printing diffs to show they don't match?

[gre]

"how to print diffs" is an interesting subject, I was first thinking it wouldn't print anything, just telling you if it matches. But maybe there is something smart to do in "how different it is" (like at least could print the file size, or how many % it changes) but i'm not sure it would be very relevant information for a binary file.

I'm not sure how easy it is to implement this in Jest but, yeah, I might try to work on this.

[ghost]

Kinda sounds similar to my request few days back https://github.com/facebook/jest/issues/1978. At the moment I'm trying some middle ground (checkout Resemble.js for image diff statistics). I'm currently trying to put together some small reports of failed snapshots and rendering the output side by side (and overlayed with a transparency slider):

[gre]

@IPWright woo interesting. does that means Jest runs on web in your case?

[ghost]

@gre no, not quite. I've hacked a couple of internals in Jest (I'm trying to get a pull request sorted but need help because I can't get the tests working properly). What I've currently got is a little primitive and of prototype quality (but we think it'll actually deliver us some value already), but works like this:

• Run Jest via a helper function which creates React fake DOM, appends my charts and runs toMatchSnapshot().
• If snapshot failed then
  
  
  
  • Update results.json fail array with the name of the failed test
  
  
  • write out an ${testname}.actual and ${testname}.expected to disk
  
  

From there you can launch a pre-made results.html which simply loads in the results.json and picks through all the failed tests. I'm hosting all this in a node web server, so It'll go request the expected/actual files and simply render the DOM side by side. Bonus is I can include any pre-compiled stylesheets at the top of the page to get some basic styling too.

[cpojer]

@IPWright please start a PR and ask questions on how to work through the issues! Happy to help.

[thymikee]

@IPWright any updates on this one?

[thymikee]

@IPWright bump 😄

[ghost]

@thymikee sorry - I don't read my Git emails by default (as I get 100's). I've got another issue https://github.com/facebook/jest/issues/2038#issuecomment-271368503 that almost allowed me to do what I wanted, but I abandoned it. I'll see if I've still got the code and resurrect it.

[IPWright83]

Seems unfortunately my approach (https://github.com/facebook/jest/issues/2038) doesn't really fit well with the Jest matchers, which was my way of trying to do it without modifying much of the Jest internals :(

[cpojer]

It seems like we aren't moving forward with this for now, so closing this issue. I'd still be excited to see what support for this may look like in Jest, though. Let us know if you are interested in picking this back up.

[FredrikNoren]

+1 on this issue. I'm generating screenshots using nightmare which I want to compare to the expected look. I've hacked something on top of Jest to accomplish this for now (using the snapshot serializer) but it's terribly hacky. I would suggest re-opening this issue.

[thymikee]

@FredrikNoren this is interesting, can send a PR so we can discuss and iterate on the idea?

[FredrikNoren]

@thymikee I only have a hack right now, I'll see if I can find some time to write a proper implementation in jest