Modules: Dynamic Modules Status

[guybedford]

Update 5/11/2018: The Dynamic Modules implementation is now complete in v8, pending further review (at https://chromium-review.googlesource.com/c/v8/v8/+/1303725). This means we can have named exports in transparent interop of ESM from CJS

As you know, I've been pushing a Dynamic Modules specification for ensuring named exports on CommonJS while retaining a correct execution order in transparent initerop. This spec has been carefully worked out at https://github.com/guybedford/proposal-dynamic-modules.

While transparent interop remains a possibility, I believe this specification and work is important to follow so we can provide the best experience to users.

Initially this was created as a Stage 1 TC39 proposal, but it turns out the model is better served by having a Node-specific module record specification implemented in V8, just like Web Assembly will be its own module specification outside of TC39.

This way only a few ECMA262 specification changes are needed to support dynamic modules - https://github.com/tc39/ecma262/pull/1306.

This PR was discussed at the latest TC39 meeting, and it was determined that it should only land following implementation work / feedback.

Thus, the next steps on Dynamic Modules here are:

1. Setup the Dynamic Modules spec as a Node.js specification moving it from my personal account to the Node.js repo account. I've requested this here - https://github.com/nodejs/admin/issues/231
2. Work with v8 to get an implementation together. I will be blocking off a full week or two as needed here to do the C++ work myself in the coming month, if anyone else is able to assist with this please let me know - it would be great to pair on the work.
3. With implementation feedback from v8, get the spec changes merged back into ECMA262.

While this work all rests on a hypothetical, I hope we can all appreciate it is an important consideration on that hypothetical, and should be pushed regardless.

[devsnek]

i'd be happy to help with the v8 impl.

[jkrems]

I've done some work on import.meta in V8 in the past. Happy to help!

[guybedford]

Thanks @jkrems, whats the best way to reach you to discuss the details further?

[jkrems]

Most likely email or GChat ([email protected]) or Twitter DM (@jankrems). The former definitely is the more reliable way of reaching me quickly.

[guybedford]

@jkrems ok I believe I've sent you an invite there, but let me know if you don't see it.

[jkrems]

I'm not seeing an invite (including in my spam). Just in case you can also try my personal email ([email protected]).

[jkrems]

Let me know if there's any communication method you'd prefer in case this doesn't work out. :)

[guybedford]

To further update on the spec and implementation here:

• I've moved the spec repo into the Node.js organization at https://github.com/nodejs/dynamic-modules.
• I've also got an early draft implementation of dynamic modules working finally in v8 at https://github.com/guybedford/v8/compare/master...guybedford:dynamic-modules

The API can be seen in the test case here using HostExecuteDynamicModuleCallback and module->SetExport.

The following implementation steps are pending:

• [x] Handle namespace creation for dynamic modules
• [x] Provide and test linking error for undefined exports on dynamic modules
• [x] Provide and test linking error in uninstantiated circular leaf edge case
• [ ] Further review on the implementation towards landing in v8

@jkrems @devsnek thanks both of you for feedback so far, do try it out again if you can!

[jkrems]

Nice work! I played around with the test case to try some more module graphs (e.g. multiple imports of the same dynamic import) and it worked without a hitch. :)

(There's a single else on its own line in api.cc but I think the V8 code formatter should be able to take care of that..?)

[devsnek]

@guybedford nice work! i have some feedback on the v8 internals that i can put up once you have a CL.

[guybedford]

@jkrems awesome to hear it didn't immediately break :)

@devsnek that would be great - feel free to leave feedback at the repo there in the mean time, as it will probably take me a few more days yet to get to items 1 - 3.

[guybedford]

Further update on dynamic modules - the implementation at https://github.com/guybedford/v8/compare/master...guybedford:dynamic-modules is now fully spec-complete, and just needs some refactoring work on the data structure and module class side.

@devsnek you mentioned you had some ideas for how to handle the script metadata part of dynamic modules - would value your thoughts on that as I'm starting work on that now.

[guybedford]

PR for review at https://github.com/guybedford/v8/pull/2. Comments welcome.

[guybedford]

The implementation of dynamic modules was finally completed to spec (and the spec worked out as written). Review at https://chromium-review.googlesource.com/c/v8/v8/+/1303725.

[MylesBorins]

I'd like us to discuss this one more time in tomorrow's meeting and ensure we have explicit consensus from our group that we want this moving forward.

[guybedford]

Sure, I'd be happy to provide an update here on how last week's TC39 went.

[guybedford]

As per the meeting, I've posted a Node.js issue with a status update at https://github.com/nodejs/node/issues/24894.