Esm: Import from web

[StevenLiekens]

I was playing around with the ES module loader + unpkg but found that it is not supported.

import { Observable } from 'https://unpkg.com/[email protected]/_esm2015/index.js';

Debugger attached.
file:///d:/src/xyz/src/main.js:1
Error [ERR_INVALID_PROTOCOL]: Protocol 'https:' not supported. Expected 'file:'
at Object.Module._extensions..js (internal/modules/cjs/loader.js:718:10)
Waiting for the debugger to disconnect..

If at all possible, please consider implementing importing from the web in the way that the browser does.

[jdalton]

Hi @StevenLiekens!

The esm loader will work with solutions that enable http requests from require. If you're up for experimenting you can tackle this with the --experimental-worker, .workerData, Atomics.wait, and Atomics.notify.

[StevenLiekens]

I'm very new to this stuff. What do you mean by "solutions that enable http requests from require"?

[jdalton]

There are ways to hook into require and customize it.

[Wildhoney]

@jdalton is it possible to force esm to transform import statements into require statements so that we can intercept the requires? As import statements cannot be caught.

[jdalton]

Hi @Wildhoney!

If you are hooking require you can catch import statements. We even have scenario tests on this. Keep in mind static imports are hoisted so you can can't mock imports in the same file you use mock-require or other require-hooking technique but subsequent modules loaded that contain static imports will be hooked.

[Wildhoney]

Thanks @jdalton although I'm finding that somewhat difficult to follow. It seems as though the __esModule is the important part, as commenting that out causes the tests to fail.

The way I have it working with resolving URL modules to their local module equivalents is by preprocessing with @babel/plugin-transform-modules-commonjs before passing the baton onto esm. I can then use my own little module for rewriting the URLs by monkey-patching the require function.

For me this approach is preferable because otherwise you need to be explicit in specifying the mocked URL imports. Also the path for URL imports can change:

https://cdn.jsdelivr.net/npm/[email protected]/src/x.js
https://cdn.jsdelivr.net/npm/[email protected]/src/y.js
https://cdn.jsdelivr.net/npm/[email protected]/src/z.js

For the above I assume we'd need to specify 3 mocks – I didn't actually get it working for URL imports, so not sure my hypothesis is correct. Instead I wanted something more automated that simply caught the URL imports – which only works with require as imports cannot be intercepted automatically — from my understanding — without manual intervention.

@StevenLiekens the way I have it working for my ava tests is by:

package.json

"ava": {
    "require": [
        "@babel/register",
        "esm",
        "node-url-imports"
    ]
}

.babelrc

{
    "ignore": [],
    "plugins": ["@babel/plugin-transform-modules-commonjs"]
}

Which essentially transforms all import statement into require statements, which allows them to be intercepted by monkey-patching require and resolving all URL imports: import x from 'https://cdn.jsdelivr.net/npm/[email protected]/src/x.js' → import x from 'a/src/x.js'.

[StevenLiekens]

Not saying what you accomplished isn't good, but now I'm only interested in a Babel-less solution. (https://github.com/standard-things/esm/issues/667#issuecomment-437686944)

[Wildhoney]

Ideally I'd want a Babel-less solutions too (from what I've witnessed over the years, Babel brings along as many problems as it fixes), but as the import statement cannot be monkey-patched, it's not possible — from what I can see — to automatically detect URL imports and rewrite them to be Node import/require compatible.

That is unless esm provides a way to force import statements to be transformed to require statements — maybe even just URL imports. Maybe @jdalton has a better solution to achieve the above solely with esm.

On the up-side, the Babel config above is very minimal and only rewrites the imports to require – nothing else. All the rest will be handled by the wonderful esm.

[jdalton]

@Wildhoney Our import uses require behind the scenes. If you have a small scenario repo I can look at I might be able to help narrow down your issue.

[Wildhoney]

Thanks @jdalton – I've setup a simple example in example/index.js.

Use the node -r esm example/index.js command.

[jdalton]

Thanks @Wildhoney!

I found the issue and am working through it on my end.

[TomasHubelbauer]

Hi, I came here looking for the same thing and I wonder if you'd be open to including http/https support by default? I am wishing for Node to behave more like Deno when it comes to imports and this would really bridge that gap. Although it's totally fair to see this as something which should remain separate.