Esm: import value is 'undefined' when using esm in an application that uses packages with esm

[CacheControl]

This is a very weird bug I encountered this week with esm. I had a package that was on esm 3.0.6, which was installed in an application using esm (same 3.0.6 version). When running tests in my application, the import value of the package _consistently_ resulted in undefined whenever run with nyc + mocha.

Thinking it was a special combination of mocha/nyc/esm, I created a repo to reproduce, and was able to get the behavior to happen with _only_ mocha (no nyc), however this time it was seemingly random and would resolve itself after running the tests a few more times.

The repo is here, and includes a screen capture of the behavior and installation instructions. It only fails for me every so often, and randomly. You'll see in the screencap I edited package.json, because at the time I observed the problem goes away after changing a local file (I thought perhaps it was some kind of caching bug), however later on I realized that running the tests over and over again would eventually result in success. It may be my imagination, but the problem seems to occur more frequently when run through mocha + nyc, though again I can make it happen with mocha alone.

edit: I've reproduced this with both a standard npm install of the esm package, and when npm linking it.

Node version: v8.9.4
npm version: 5.7.1

[jdalton]

Hi @CacheControl!

I can't seem to reproduce this using Node v8.10.0 (npm v5.7.1).
Are you on Windows?

Update:

I tested this on Mac OS and Windows 10 with Node v8.10.0 on both without issue
(though on Windows I removed the .npmrc file).

It looks like your example isn't really using nyc.
Is it missing some parts to be more representative of your issue?

[CacheControl]

@jdalton I'm on OS X El Capitan (10.11.6)

nyc isn't required to reproduce the issue (can do with a simple npm test, though I seem to able to make it happen more consistently when running nyc npm test.

I committed that repo as-is immediately after taking the screencap

[jdalton]

Ah nice, I got it on my second run with npx nyc npm test. Digging in!

Update:

This is interesting... the instrumented files by esm aren't changing from run to run so it's something else. What is loading is changing though. In working cases more is loaded. I see it fail one run, succeed another, fail the next, succeed after that, and so on.

Update:

It looks like something odd when reading the cache folder via readdir. Still digging...

Update:

Ok, it's the data map in the .data.json that is changing between runs. That data is what informs the loader of what is or isn't exported which is why when it's missing an entry it's reporting undefined. Now to find out why it's toggling state.

Update:

Ok, it's not merging old state with new state correctly.

Update:

Fixed it! Now to make super sure!

Update:

Patched https://github.com/standard-things/esm/commit/1c93540010a05fd213b740a19f0f003640753377 and https://github.com/standard-things/esm/commit/6b8a60522dc26ec87dfe9b5d3e9bcd6bd8d49b99.

Besides fixing the issue this avoids parsing files known to have no transformations and makes the cache less chatty too 😋

Update:

v3.0.8 is released 🎉

[jdalton]

@CacheControl

This scenario caught 2 issues.
Would you be up for creating a simplified scenario test for esm?

[CacheControl]

@jdalton heck yes I'm up for it!