Deno: import.meta.resolve implementation

[guybedford]

Node.js has been shipping an import.meta.resolve implementation for some time behind the --experimental-import-meta-resolve flag described briefly here - https://nodejs.org/dist/latest-v14.x/docs/api/esm.html#esm_no_require_resolve.

In Deno such a function could resolve the local import map when set including applying scope resolution based on the current module context.

Is there interest from Deno in implementing this API? I don't think it would be sensible for Node.js to unflag this feature without getting wider support, but Deno is likely the only other environment that might benefit from this. Feedback would help us to consider the unflagging process.

[nayeemrmn]

We shouldn't implement this unless browsers do. Also I don't think it's possible to make this work with bundling... not in a way that's consistent with our handling of other import.meta fields. It would have to know the full URL at bundle-time for import map resolution. Would it look up the import map at runtime?

[guybedford]

The model for require.resolve in bundles we use in ncc is to reemit the asset or folder that was resolved based on static analysis of the expression flow and then to replace a direct asset reference into that expression.

The same model can apply to import.meta.resolve nicely because it is exactly a statically analyzable construct.

Browsers have no driving need to implement this, so if we (Node.js and Deno) wait, we may wait years. I'm happy to leave this open to track further though.

[lucacasonato]

This looks like something that can be fully implemented in userland with import.meta.url in Deno.

We don't have node resolution, so I guess the implementation would more or less look like this: import.meta.resolve = (path, base = import.meta.url) => new URL(path, base).toString();.

It also does not really make sense for this to return a Promise<string> in Deno because we can resolve this synchronously without IO ops, just like the browser. But I guess Node would require this to be async...

[bartlomieju]

That method should probably take into account currently loaded import map, so it would actually be an op.

[lucacasonato]

Ah right - import maps... :facepalm:

[nayeemrmn]

Yep, Deno.resolveModule(fullyQualifiedUrl) since it's a host environment thing. Or more likely wait for the web to have something similar. And if they implement that in tc39 import.meta as opposed to a global web API, we would too.

[kitsonk]

TC39 doesn't deal with things like module resolution. import.meta was specifically designed to deal with the stuff TC39 doesn't. It is WhatWG that has attempted to move forward on things like this, providing import maps in the first place.

@guybedford is there no interest in WhatWG that you are aware of? I know module loading and resolution is never a straightforward topic! 😉

[kitsonk]

(Also for the record, import.meta was adopted by TC39 for some of these very specific reasons: https://github.com/tc39/proposal-import-meta. It is a "catch all" for runtime specific implementations. So alignment between Deno and Node.js makes a lot more sense than throwing it into a truly unique namespace like Deno IMO)

[guybedford]

The benefit of import.meta.resolve is having a standard resolve function which build tools can statically analyze regardless of platform to eg perform asset relocation (https://github.com/vercel/webpack-asset-relocator-loader).

If Node.js and Deno have their own custom functions and encourage that, those patterns get engrained in usage and users have code that does conditional checks and calls these different functions - the build tooling to deal with bundling while retaining asset references becomes very specific to each platform then or needs lots of custom work.

The custom usage for a library then looks something like:

function crossPlatformResolve (specifier) {
  if (typeof Deno !== 'undefined') {
    return Deno.resolve(specifier, import.meta.url);
  } else if (typeof process !== 'undefined' && process.platform.node) {
    // nodejs has yet to specify such a custom resolver too
    return require('module').nodeResolve(specifier, import.meta.url);
  } else {
    // etc for all other platforms
    throw new Error('no resolver found');
  }
}

const template = await (await fetch(crossPlatformResolve('pkg/template.html')).text();

when it could just have been:

const template = await (await fetch(import.meta.resolve('pkg/template.html')).text();

which systems like asset relocators can then statically replace with:

const template =  await (await fetch(import.meta.url + '../path-to-inlined-template.html'));

as a pattern for bundling asset references.

The alternative of custom function patterns without statically analyzable assets being easy becomes the path the longer we wait, and it's fine, but we can just do much better if we can move now.

As mentioned, I feel that environments like Deno and Node.js have the driving use cases here, the browser doesn't necessarily. Waiting on WhatWG means giving up agency, as neither Deno nor Node.js have any voice at WhatWG.

There was a WhatWG PR for this in https://github.com/whatwg/html/pull/5572, the main contentions there were whether it should be sync or async, and whether there should be a second argument or not.

To give you the gist of the current situation:

1. If both Node.js and Deno were to decide that a sync import.meta.resolve function without a second argument is fine, that would get us alignment with Domenic and a potential path to move forward immediately
2. Alternatively, both platforms can choose to collaborate on driving work here with the API that suits these platforms best.
3. Avoid scary decisions entirely and do nothing.

I understand completely (3) is the easy status quo though :)

[timreichen]

@guybedford I would vote against implementing a non-standard like that.
Agree with @nayeemrmn that if this was implemented it would be better to wait or implement it as a function in std/node since it is kinda node specific.

[lucacasonato]

@timreichen I didn't say this should be implemented in the Deno namespace. And also I disagree that this is node specific. This problem is something that applies to Deno too. Only the sync vs async implementation is where we diverge from Dominic's proposal.

[timreichen]

@lucacasonato sorry read that wrong. Corrected.

[nayeemrmn]

Re sync vs async, the permissions API would be a precedent for us going async to match the standard even though the underlying calls are sync. I don't see what reason anyone would have to ever "expect" a sync API without concerning themselves with the implementation.

[kitsonk]

Well, we in theory want to be able to resolve async, especially for remote modules. This is where the semantics differ from Node.js in that would we return the initial URL for a module, or the final redirected URL for a module?

All this being said, @bartlomieju and I are embarking on some major refactors of how we do dependency analysis determining what processing or not needs to be done to modules. It feels like this is something we might want to consider after the dust settles on that.

[guybedford]

Thanks for the feedback here, it does sound like those refactorings / resolver concerns in Deno need to be fleshed out more in order to determine what is best for Deno here further with regards to APIs here.

It would be great to work towards alignment between Node.js and Deno when those directions are ready, and I will continue to reiterate that specifications are secondary to implementations, and in this space Node.js and Deno have the interest end engagement of the implementation / ecosystem conventions first and foremost. There is certainly risk diverging from browsers yes, but if we were to eg jointly propose such browser specs as part of the implementation / unflagging process I'm sure it would be well received. I'd be more than happy to collaborate on specification work here as well.