Modules: Relaxing the '#' restriction on imports

[guybedford]

Initially we added the # prefix restriction to package imports in order to ensure that it was always well-defined when a private local mapping might be applying.

I just came across a real use case for Web Assembly where it could be worthwhile to relax this restriction.

I explain the use case in detail below, but this seems a compelling reason to relax the restriction to me. I was wondering if anyone has concerns around relaxing this restriction?

Consider a WASI module that does something like (Web Assembly example as if it were written as a JS module):

app.wasm

import { fd_write } from 'wasi_snapshot_preview1';
export memory;
export function _start () {
  fd_write(1, 0, 0, 10);
}

We can execute this in Node.js today using the WASI core module.

If we wanted to import this module using the upcoming ESM integration, a package author might be able to write:

import * as app from './app.wasm';
import { WASI } from 'wasi';
new Wasi().start(app);

The problem with the above is that the wasi_snapshot_preview1 import is supposed to provide an fd_write function that is implicitly bound to the exact memory export instance from the Wasm module. What this means is that it cannot be a shared library in node_modules, it must be a local ./local-wasi-runtime.js file that represents the local runtime virtualization just for this package.

That is, for this workflow to work, we need the ability to map import 'wasi_snapshot_preview1' into import './local-wasi-runtime.js' inside of the Wasm binary imports itself.

Package imports exactly provide this functionality if we could permit the import mapping:

{
  "imports": {
    "wasi_snapshot_preview1": "./local-wasi-runtime.js"
  }
}

which exactly could be supported if we relax the # prefix restriction.

An alternative here might be to try and convince Wasm to use this # prefix to permit virtualization in turn and allow these use cases.

[ljharb]

The alternative seems better; by removing the # prefix, it expands it from "internal imports" to "package-scoped overrides", which is a very very large can of worms to open.

[guybedford]

@ljharb the thing is, it is very hard to ask Web Assembly developers to very nicely get LLVM and WASI to change the default conventions to support such a thing!

The alternative may be not being able to import WASI using the ESM integration...

[guybedford]

Actually package name self-resolution could provide a route here too --- have a subfolder with a package.json "name" field equal to "wasi_snapshot_preview1" which then has a main pointing to the local wasi instance.

[robpalme]

Letting # naturally become best practice sounds ok to me. We can obviously do more to communicate this over time.

IMO this relaxation would harm that adoption story because it makes it less obvious as to whether a package is following these good conventions.

[guybedford]

Thanks @ljharb @robpalme, helps to know. I have started the conversations with some Wasm folks to try and encourage having LLVM emit "#env" / "#wasi_snapshot_preview1" etc style specifiers. I am not even slightly hopeful I will make any progress in convincing anyone here though :)

Alternatives are to have LLVM / toolchain permit alternative names for these, rebuild Wasm when publishing it into the JS ecosystem or to have the JS esm integration return uninstantiated modules for a more flexible instantiation process. Will follow up appropriately.

[guybedford]

Just thinking about another use case here - there is another argument that disabling this restriction could enable self-contained monorepos. I wonder if anyone else has considered this use case too @ljharb @robpalme?

[ljharb]

What would it enable that they can't do now by using #-prefixed paths throughout the monorepo?

iow, many monorepos make it explicit and clear when a specifier is "inside the monorepo" by prefixing it like @/blah or ~/blah, etc, and transforming those specifiers to relative paths. In this case, you could do the same with #blah at the root, no?

[guybedford]

Thanks for the response - yes I was exactly thinking the explicit # for an
internal pattern is actually clearer too.

Just trying this out on a package myself where I have separate sub packages
and really liking the pattern so far of having a single ‘interface’
package.json.

On Mon, Sep 14, 2020 at 16:48 Jordan Harband [e-mail]
wrote:

>
>

What would it enable that they can't do now by using #-prefixed paths
throughout the monorepo?

iow, many monorepos make it explicit and clear when a specifier is "inside
the monorepo" by prefixing it like @/blah or ~/blah, etc, and
transforming those specifiers to relative paths. In this case, you could do
the same with #blah at the root, no?

—
You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub
https://github.com/nodejs/modules/issues/551#issuecomment-692377155, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAESFSXM55SXIZH2ZY65C43SF2TWVANCNFSM4Q5HSV2Q
.