Microbundle: Find an alternative to snapshotting the build output

[Xiphe]

As I'm trying to fix tests for https://github.com/developit/microbundle/pull/478 I found that the snapshot tests of minified & compiled code are not at all human-friendly and pretty frustrating to debug.

What do you think of replacing them with higher level tests that do not compare char-by char but instead test more meaningful things. For example if the compiled code actually works as expected + uses the correct module format + has a sane size.

I'd be happy to provide a more detailed example on how this could look like if you're interested.

[Xiphe]

git clone [email protected]:developit/microbundle.git
cd microbundle
npm install
npm test

Gives me this:

(currently on node@12 / [email protected])

[schmod]

what if we run the source through prettier before comparing the diffs?

[Xiphe]

I don't think prettier would normalise the variable names.

Even if we manage to stabilise the build artefacts so that they don't change that often I wanted to raise the question if the tests are testing the right thing. Right now a majority of the assertions are testing implementation details of rollup.

[developit]

One option would be to apply a cascading rename via a Babel plugin. That would be relatively simple to do, and would guarantee that structurally-equivalent code would also be syntactically-equivalent.

[developit]

FWIW the snapshot issue is something to do with Linux VS MacOS - CI and folks with linux workstations get results that match the snapshots, but us MacOS folks end up with the weird deltas. It wasn't previously an issue, which makes me think this is a bug in one of our dependencies (very likely Terser).

[Xiphe]

I'll dig deeper into that next time i work on the project.

[ForsakenHarmony]

The raw file contents were added so we could see what actually changed, not just the size

[Xiphe]

@ForsakenHarmony do you mean we therefore should keep it that way? My opinion is that microbundle should not care about an additional bracket.

[Xiphe]

Since we have a package-lock.json I don't experience a snapshot mismatch between the ci and my machine.

Though my opinion is still that the snapshots are too detailed i also assume that creating a test-strategy that is closer to the api surface of microbundle will introduce a lot more complexity in the test-setup.

Will therefore close this for now.

[ForsakenHarmony]

Just hard to find an alternative, if the size of an output changes we want to know why

Not sure how to improve on that, if you have any ideas, feel free to let us know

The mismatch is quite annoying, seems to happen mostly on macos, not sure what's causing it