Modules: Feature: Retrievable module metadata

[GeoffreyBooth]

It is possible to retrieve metadata about a module being imported via an import statement.

Use case 13.

[Jamesernator]

I'm not sure what this feature is. Is this just asking for capabilities like import.meta or is it asking for the ability to get metadata about another module?

If it's the later this could (perhaps weirdly) be solved by import.meta again by exposing a function to get the metadata for another module e.g.:

// Would get the metadata of another file resolved
// in the same way as import() 
import.meta.metadataFor('./foo.mjs');

[benjamingr]

Hey, I think you might be missing the use cases document - https://docs.google.com/document/d/10BBsIqdAXB9JR2KUzQGYbCiVugYBnxE4REBakX29yyo/edit

This use case by @jkrems

Robin reads the source of one of their projects and finds an import statement. They want to quickly find the file being imported. After a few steps they find the imported file in their file system.
(q: Is this runtime? Debug? Static? require.resolve)

This can for example be done by import.meta.url or other stuff (so your assumption was correct).

Please don't hesitate to speak up if there is anything unclear about the discussion here.

[ljharb]

import.meta.resolve seems like it might be particularly helpful.

[jkrems]

They want to quickly find the file being imported. After a few steps they find the imported file in their file system.

The use case as written was about everyday reading/editing of the code. E.g. it was about a person reading the code being able to quickly find the file if they'd switch to a terminal or a file browser. With browser ESM that's super easy (right now) but CommonJS scores much lower here.

[SMotaal]

Sorry for going off on a tangent and in detail

I found that part very interesting as there is possibly unique opportunities to be considered:

(q: Is this runtime? Debug? Static? require.resolve)

Normally, module resolution systems are either a) browser (per spec) or node (per config) runtime or b) runtime-emulating with or without a mapping layer for development. In other words, at the core of every non-runtime module resolution implementation is some logic that emulates to some degree the target runtime, and those implementations may or may not include additional layers for mapping between source and target specifiers.

Given that most development workflows today involve at least one such implementation. Add to that the fact that each implementation (even if flawless) often involves additional configuration points. It is easy to imagine how all these different systems exponentially complicate the development experience, limit it or sometimes even break previously existing projects. Just consider how the adoption of something as widely accepted as ES modules has been impacted across the various tools.

Node is in a very unique position at the moment when redesigning the module system in that it can take over the most basic module resolution layers and provide a Module Resolution API shared across runtime and development use cases. Essentially, the module system would be clearly broken down to a) resolution and b) evaluation or execution. The latter subsystem is strictly employed by Node's runtime, where it would depend on a single instance of the former following the specific runtime target configuration.

In that sense, I can imagine that other node-based development tools (ie Babel, CoffeeScript, TypeScript... etc) can also create separate instances of the Module Resolver with a specific configuration profile for some runtime target. And at the same time, debugger simply use the same instance of the Module Resolver used by Node's runtime or in more exotic cases simply decorate or extend that instance which can be useful for overloading, hot reloading, or VM.

This divide can lend itself very nicely across a number of features.

[GeoffreyBooth]

Node is in a very unique position at the moment when redesigning the module system in that it can take over the most basic module resolution layers and provide a Module Resolution API shared across runtime and development use cases.

This sounds like a great suggestion. Do you mind adding one or more use cases to the use cases doc that cover this? And in the meeting, assuming a feature gets created to correspond with those use cases, this can get is own feature issue.

[GeoffreyBooth]

The use case as written was about everyday reading/editing of the code.

@jkrems That actually ended up as a separate feature: #103. That’s just how it was in the features document; in discussion I guess the one use case got spun out into two features. Maybe this one wasn’t intended from the use case, but it seems a reasonable feature request on its own.

[SMotaal]

@GeoffreyBooth after the much needed hours to catch up on what I missed over the past three weeks (away due to family emergency) I just added 4 use cases that kind of beg the feature — it's hard to not be biased when you have feature in mind.

And in the meeting, assuming a feature gets created to correspond with those use cases, this can get is own feature issue

So what's next? I have dedicated time today and really want to get back on track with our efforts.

[bmeck]

If we put something on import.meta that doesn't match with browsers I would request that we namespace it to something that browsers can agree not to use like import.meta.node.resolve if import.meta.resolve cannot be isomorphic.

[benjamingr]

@bmeck it would be way cooler if we could make .resolve work on the client - I can totally see the appeal in that.

If we can't, then if we go with import.meta.require then import.meta.require.resolve might be appropriate.

[ljharb]

With or without transparent interop, fwiw, import.meta.require would still be very useful (because the language lacks a way to synchronously import things conditionally and/or dynamically), so import.meta.require.resolve also would make sense

[benjamingr]

With or without transparent interop, fwiw, import.meta.require would still be very useful (because the language lacks a way to synchronously import things conditionally and/or dynamically)

Given top-level-await just made stage 2 I don't really see this as a problem.

I realize I changed my mind from last week - but then again last week I didn't know it was making progress and this week it's stage 2 so there's that.

[ljharb]

Top-level await doesn't change this discussion in the slightest, imo - TLA is no different than a setTimeout that reassigns a property on module.exports or that reassigns an exported let.

[benjamingr]

@ljharb I'm sorry, I don't understand that - would you mind explaining or showing some code that demonstrates this issue?

[ljharb]

@benjamingr

// CJS
module.exports.foo = 3;
module.exports.bar = undefined;
fetch('some JS file').then(() => {
  module.exports.bar = 1;
  module.exports.foo = 4;
});

// ESM without TLA
export let foo = 3;
export let bar;
fetch('some JS file').then(() => {
  bar = 1;
  foo = 4;
});

// ESM with TLA
export let foo = 3;
await fetch('some JS file');
export let bar = 1; // this will be hoisted above the `await`, but not evaluated until here
foo = 4;

All three of these modules start out exporting "foo" set to 3, and once the JS file is fetched, change to export it set to 4. Adding top-level await doesn't have any impact on blocking, or on mutable exports, or on the shape of module.exports or the module namespace object.

[targos]

@ljharb

As I understand how top-level await should work, it is different because the importer of foo will never observe the original value 3. Am I wrong?

[ljharb]

@targos you are wrong; the instant the TLA occurs, the importer resumes, so they will see identical behavior in all three cases.

[benjamingr]

All three of these modules start out exporting "foo" set to 3, and once the JS file is fetched, change to export it set to 4. Adding top-level await doesn't have any impact on blocking, or on mutable exports, or on the shape of module.exports or the module namespace object.

So the request here is explicitly for importing modules completely synchronously?

Would you mind motivating this with a real-world use case? I think that TLA addresses the use cases we discussed so far. Maybe I just missed one.

If there is another use case that requires the synchronous behavior we should add it to the list.

[ljharb]

@benjamingr there's already use cases in the doc that require synchronous import of CJS as a part of transparent interop; this specific issue i believe would be addressed by import.meta.require.resolve.

[MylesBorins]

I just followed up with @domenic and the current intention of the TLA proposal is that the parent module would not execute until after the child resolved

In the below case, the module would log 4, not 3

import { foo } from './dep';
console.log(foo);

dep.js:

export let foo = 3;
await fetch('some JS file');
foo = 4;

[ljharb]

@MylesBorins that's very confusing to me, and that's something we need to address separately in that proposal.

[bmeck]

@ljharb evaluation from TLA blocks dependents until end of source text is reached. It does not let dependents evaluate immediately after the first await is reached.

[giltayar]

@ljharb - I believe top-level await significantly changes the ESM landscape (funny how the feature is "just" called "top-level await"). And all because of _default export_. Assume a module with this code (and assume fetch is like the browser fetch):

export default await fetch('http://....').then(response => response.text())

This module MUST be loaded asynchronously—and all of it's top-level await-s waited upon—before continuing with the parent module execution.

Why does this significantly change the ESM landscape? Because, from what I can see, this module cannot be transpiled to CJS!. No amount of babeling will make this work.

Top-level await transforms ESM from "could be synchronous" to "must be asynchronous". And this is huge. It is _finally_ realising the benefits of ESM's asynchronicity.

[benjamingr]

This module MUST be loaded asynchronously—and all of it's top-level await-s waited upon—before continuing with the parent module execution.

Are you sure this wouldn't transpile to exporting undefined (or a TDZ) and then replacing the value of the live-binding with the value of the response text?

[ljharb]

TLA advanced to stage 2 prior to my having this understanding; I’ll revisit this with the committee next time it comes up.

[jdalton]

This thread morphed into a bit more on TLA. We should open a new issue to discuss TLA specifically or move discussion to the TLA repo.

[giltayar]

Are you sure this wouldn't transpile to exporting undefined (or a TDZ) and then replacing the value of the live-binding with the value of the response text?

Yes, I asked @MylesBorins about it in one of the modules meetings, when we discussed top-level await, because I knew that the answer to that would determine whether it was just top level await, or a whole breaking change from the babel-model of synchronous loading.

[guybedford]

Could we change the title of this feature here to "Provide resolver API" perhaps? As it sounds like it is being confused with module metadata which actually doesn't seem to have its own feature issue currently.

[guybedford]

This seems like a duplicate of #103 to me, as they both reference the same use case?

[demurgos]

See https://github.com/nodejs/modules/issues/103#issuecomment-405901273

This issue is about getting programmatic access to metadata of imported modules.

103 is about human developers being able to reproduce the resolution algorithm.

[guybedford]

@demurgos if you look at the original post of both, they come from the same use case. The exact text of the use case is copied in https://github.com/nodejs/modules/issues/104#issuecomment-390638078.