Tape: author tape itself as ES Modules

[lastmjs]

I'm wondering if plans could be made to convert tape to use ES modules, which finally bring a universal module format to JavaScript engines, both in the browser and in Node.js. Right now I am personally having difficulties using tape in the browser because of its use of require statements, and my use of a build system that assumes ES modules. Ideally no build system would be necessary and ES modules seems like a great candidate for allowing that.

[ljharb]

1. node has not yet shipped ESM support unflagged; until it does, it's not appropriate to ship ESM with any package.
2. I'm not sure what "WAS modules" are, but I'm assuming you mean "ES modules"
3. Every package that uses require should be bundled.

A build system is, and always will be, necessary - when node has ESM support, packages that ship it will be using non-URLs in their import statements, which browsers won't understand - you will still always require a build process to transform those to URLs, or to bundle them up.

[backspaces]

How about a tutorial or documentation on using tape in the browser? .. which has es6 module support thus solves this, I think. (It's probably out there but I haven't seen it yet)

I currently use Rollup to build .cjs files so can test with that, but I foresee the need to run tape in the browser.

[ljharb]

ES Modules are only in two browsers atm, and you'd need a build process regardless to convert all your import paths to URLs, so it's simply not going to be realistic imo to ever have no build process. As such, any time you run tape in the browser, you'd run it on transpiled ESM - ie, CJS.

[backspaces]

Yeah, I've got to Rollup everything to a billion formats so no big deal.

But Tape's readme seys "browser". So if you run in the browser, write it up and I'll use one of the (many more than 2) browsers that deal with modules. Safari, Safari Preview, Echo, Chrome, Canary, Brave, .... And yes, the other need flags, but this is TESTING .. i.e is not in the user's way, only the development. All our team has a testing stunt for a "modern" browser.

This is all beside the point. Which is: Could you document clearly how Tape runs in the browser?

[ljharb]

Run it through browserify or any similar tool (webpack, rollup, etc) and it should work fine in a browser.

[backspaces]

Run Tape through browserify? Or do you mean add browserify to my workflow in some way that makes it work with Tape in the browser better?

My existing dist/ dir has three Rollup bundles: IIFE for script tags, es6 for a single file modern repo, and a cjs for node use. So my bet that is good enough for Tape use.

I hate to be ignorant, but the JS ecology is so complicated, and in some ways idiotic, it's hard for me to know just what it means to "run it through browserify"!

Thanks you for your prompt response, btw. And naturally thanks for the help in a difficult area.

[ljharb]

Run your tests' entry point through browserify, which will include a require('tape') somewhere in there.

[bennypowers]

Is https://github.com/standard-things/esm an option?

[ljharb]

@bennypowers you could certainly use that.

[bennypowers]

Just to be clear: there's no drop in solution for using the esm package - test authors would need to add the following to their test entrypoint

// Set options as a parameter, environment variable, or rc file.
require = require("esm")(module/*, options*/)
module.exports = require("./main.js")

is that what you meant, or am I misunderstanding?

[ljharb]

node --require=esm path/to/tape/test.js?

[PhilT]

tape -r esm test/**/*.mjs worked for me.

[OKNoah]

If I'm understanding the issue at a glance, it has to do with web (browser)? What about node, is there a way to pass --experimental-modules to tape as you would do node --experimental-modules?

[ljharb]

@OKNoah there’s not, currently, although that seems like a reasonable feature request.

[backspaces]

Actually, I've gotten around this issue using Puppeteer for all my tests (where needed). It's gotten pretty mature from its earlier wild west days.

Re: es6 modules https://caniuse.com/#feat=es6-module shows the 6 my team cares about having module support. We're lucky being somewhat bleeding edge I guess. We no longer use non-module libraries.

We are helping projects we depend on be es6 ready.

• Three.js: three.module.js already exists, only the examples/ are still es5 .. which include controllers we need. For those, several of us had written scripts to convert them to es6. And the most active Three issue is how to manage conversion of these to umd/esm pairs.
• Two.js: only needed the package.json help below, and are releasing that form next publish.
• dat.gui: I looked at the code and found out it was nearly there and just needed 4 changes. They did and we're go.
• d3: already there and fine grained: 30 sub-repos! And taught us the unpkg trick of having a single npm package export both UMD and ESM via the package.json module: xx field.
• hard-cases: we use a "wrapper" around es5 to convert them to a module while helping them refactor to ESM src/ files with Rollups to UMD and ESM pairs, using the package.json stunt for access to both via unpkg & ?module. Here are somewhat old references:
  https://medium.com/@backspaces/es6-modules-part-1-migration-strategy-a48de0b7f112
  https://medium.com/@backspaces/es6-modules-part-2-libs-wrap-em-up-8715e116d690

[ljharb]

@backspaces .module.js isn't a thing; .mjs is the upcoming standard in node. The "module" field is nonstandard, and will be unnecessary the instant node ships an unflagged modules implementation, so you'll want to have .mjs files ready for when that happens.

Separately, although there's a "package name maps" proposal for browsers, until it lands, for any ESM modules to run in both node and browsers without a build step is effectively impossible.

[backspaces]

Sweet! Could you do us all a favor? When you do the conversion, write it up on medium or in the project wiki?

I ask because it really is hard for folks to grasp yet-another technology and it would good to help them out.

[ljharb]

@backspaces i'm not sure what "conversion" you mean; the instant node supports ES modules, tape will. As for users; any file with .js is a normal CJS module, and .mjs is ESM.

[kevinpeno]

In case someone stumbles upon this, passing --experimental-modules to tape works fine on windows (*nix not available or I'd test). Even in a context such as this: npx tape --experimental-modules test/**/*.mjs.

[ljharb]

That definitely will work, but note the “experimental” part - it is highly likely to break over time.

[kevinpeno]

Of course! I take the risk using anything experimental. 😀

I mostly made the comment because as I was reading through it was requested that the ability to do so be added to tape. The comment was confirmation that the functionality already exists.

[lastmjs]

It's not that tape won't work in Node with ES modules enabled, it will because node is still using its CommonJS variant as well as ES modules. The point of this issue is to author tape with ES modules, so that the source code itself is written with ES modules, and then it will work natively in browsers and Node with translation of bare specifiers, once modules are no longer experimental in Node.

[ljharb]

It won’t do that until node’s module system is finalized - and either way, it’s simply not practical to aim for a world without a build process imo.

[bennypowers]

FWIW .mjs is not 100% accepted yet, afaik

[ljharb]

Yes, it is. I’m on the modules working group and although other mechanisms may be implemented, there’s basically zero chance mjs won’t be the base disambiguation mechanism.

[cowwoc]

2019 is almost over. What's the status of this issue?

[ljharb]

@cowwoc the date is irrelevant; until node ships ESM unflagged, there's no change in status.

[cowwoc]

@ljharb Thank you for the clarification. Per https://2ality.com/2018/12/nodejs-esm-phases.html#when-can-i-use-es-modules-on-node.js%3F this will happen on April 2020.

Are you saying you don't plan to begin working on this feature until then? If so, we will be waiting longer than April 2020 for this feature. Would it be possible to begin working on this feature ahead of time with an estimated delivery date shortly after April 2020?

[ljharb]

That's a bit out of date; it may happen sooner, but yes, it means the feature won't be worked on until then.

Even if this project is native ESM, though, you still won't be able to use it without a bundler (or without the project using one in advance), because it's a node module. This will be true of virtually everything on npm. You can use tape in a browser now with the same bundlers that have worked for years; there's zero reason anyone needs to wait on tape - or anything else - being native ESM.

[cowwoc]

@ljharb I'm trying to run tests written in ES6 using tape in npm (no browser involvement). What's the status of this kind of deployment? Should I be able to do this today?

[ljharb]

In node? tape being written in ESM has zero impact on that. Since node itself can’t handle ES Modules unflagged, you need to be using babel - either in advance, or via babel/register - just like you would with every single other test runner.

[cowwoc]

@ljharb So in theory, if (today) I run tape with node --experimental-modules then I should be able to write my tests in ESM and test code that itself is ESM?

[ljharb]

Yes.

This issue is about tape itself being authored in ESM - it never has nor ever will have any bearing on how your tests are authored.

[cowwoc]

@ljharb Okay. Thank you for the clarification.

[lastmjs]

Even if this project is native ESM, though, you still won't be able to use it without a bundler (or without the project using one in advance), because it's a node module. This will be true of virtually everything on npm. You can use tape in a browser now with the same bundlers that have worked for years; there's zero reason anyone needs to wait on tape - or anything else - being native ESM.

I do not believe this statement is true. There is a standard in progress called import maps. I've used them experimentally in Chrome, and they work. This should allow users to use code authored with ES modules without a bundler. The only build step required for vanilla JS code would be to generate the import map, which can be done with a simple npm script after running an npm install.

More info: https://github.com/WICG/import-maps

[ljharb]

That presumes the modules are using functionality that is all present in the browser, or imported but shimmable in the browser, and doesn’t use anything globally available in node including environment variables. I’d be interested to see a study done on npm for how many things would qualify, but i suspect it won’t be that many.

[Raynos]

One of the strong advantages of tape is that it works with ES3/5. It also works with older versions of nodejs.

This makes it possible to test older versions for legacy back compat.

It would be a shame to lose that advantage.

[ljharb]

We’d never switch wholesale to ESM for that reason; but if there’s a way to do dual packages, it’d be fine to ship both implementations.

[cowwoc]

There is always a way, using rollup.js, parcel.js or equivalent.

[ljharb]

dual packages means that consumers can require or import the same specifier and get the appropriate implementation; if you’re willing to use a bundler this issue’s been irrelevant for a great many years.

[cowwoc]

@ljharb Correct me if I'm wrong, but I believe you can easily go from ES modules to AMD and older formats but not the other way around.

[ljharb]

You can easily go from ESM to CJS, which is what the majority of the ecosystem has been doing with Babel for many many years. AMD isn’t a factor.

[donmccurdy]

Since it looks like I'm not the only one finding this issue when trying to get Tape tests on code containing ES modules to work, here's a separate issue for that: https://github.com/substack/tape/issues/514