Modules: Online Module Summit 2018-04-03

[MylesBorins]

Time

Decided via doodle

UTC Tue 03-Apr-2018 14:00 (02:00 PM):

| Timezone | Date/Time |
|---------------|-----------------------|
| US / Pacific | Tue 03-Apr-2018 07:00 (07:00 AM) |
| US / Mountain | Tue 03-Apr-2018 08:00 (08:00 AM) |
| US / Central | Tue 03-Apr-2018 09:00 (09:00 AM) |
| US / Eastern | Tue 03-Apr-2018 10:00 (10:00 AM) |
| London | Tue 03-Apr-2018 15:00 (03:00 PM) |
| Amsterdam | Tue 03-Apr-2018 16:00 (04:00 PM) |
| Moscow | Tue 03-Apr-2018 17:00 (05:00 PM) |
| Chennai | Tue 03-Apr-2018 19:30 (07:30 PM) |
| Hangzhou | Tue 03-Apr-2018 22:00 (10:00 PM) |
| Tokyo | Tue 03-Apr-2018 23:00 (11:00 PM) |
| Sydney | Wed 04-Apr-2018 00:00 (00:00 AM) |

Or in your local time:

• http://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Module+Summit+2018-04-03&iso=20180403T14
• or http://www.wolframalpha.com/input/?i=02PM+UTC%2C+Apr+03%2C+2018+in+local+time

Proposed schedule

https://docs.google.com/document/d/1oAvF_qukb1dIUWji9GjHNCL5-0uJ3_M-yPG-lzJVBbs/edit#

Joining the Call

Members of the team can join the zoom:

https://zoom.us/j/486767135

Streaming

Part 1: http://www.youtube.com/watch?v=O-7rXALxL9Y
Part 2: https://www.youtube.com/watch?v=Eaofz2Cj9Lw

---

So there is a LOT of context to the module implementation. I think it would be useful for us to schedule a day of content to document the current state of things and to get individuals who have not been as involved up to speed. I'm imaging we likely could co-ordinate a day of content around this. Please use reactions to signify interest and comment topics you would like to see or volunteer to provide. We can also invite guests from outside the group (such as TC39, webpack, or participants from Frameworks)

I'll keep this original issue updated with content as it develops.

Topics

• The history of ESModules
• Differences between ESM + Common.js
• Current State of Node.js implementation
• What is an APM and why it matters

[MylesBorins]

Would people find a "participating in open source" or "how the node project works" useful?

[giltayar]

Given that I have both a presentation and a blog post about the _current_ state of things, I can volunteer to do a presentation about that (i.e. current state, and differences between ESM and CJS).

[giltayar]

Another topic to add to this:

• The current proposals for NodeJS modules (e.g. the npm proposal and the loader:esm proposal), and how they affect things like ESM-CJS interop, migration from CJS to ESM, browser compatibility and interoperability, etc.

[weswigham]

In no particular order, these are all the (very specific) questions I have about the current state of affairs:

• Why does this need to ship in Node 10? I understand the LTS cycle and the desire to see this done, but this group is only being formed in Feb. - 3 months feels like a short time to come to consensus on and implement all semver-major changes - and shipping something that could ultimately be substandard should be unacceptable. Whatever is shipped will be here _forever_ (or close to it) - the time should be taken to get it right. I understand the desire to see it shipped unflagged (and I would, too), but I don't think it should be a hard requirement if we're not satisfied.
• Who are the abstract stakeholders for this feature? What are each group's needs and how do each of these groups needs differ? Which groups are the _most_ important to satisfy with this feature? Off the top of my head, I've got the following: Library authors (utilities, typically cross-platform, sometimes platform-specific), framework authors (front-end, back-end, fullstack - each have different desires), application authors (again, front-end, back-end, and fullstack), and transpiler or tool authors (since they end up needing to paper over the differences in any of the platforms and runtimes that surface).
• Who are the concrete stakeholders for this feature (is it everyone who was invited to this group? 😛 )? Who's already committed to migrating to node's esmodule implementation when it comes out, and who's committed to starting new projects with it? What are _their_ concerns? Their needs? Their reasons for moving in the first place?
• What, specifically, is going to make this feature more attractive than a transpilation step to commonjs, especially if running on multiple platforms is going to need a buildstep, anyway? This answer is likely different for each of the stakeholder groups identified above.
• What upcoming es features, if any, end up affecting this? How do we reconcile future syntactic changes to modules in the spec (like new export or import forms) with a desire for a stable, broadly compatible platform?

And more in the technical space of the current state of affairs:

• Does module execution order have to be maintained compared to what the user wrote? If interop allows it and they have commonjs and esm imports, what about then? (It seems like the answers are yes and no, respectively, based on the current state of things, but I'd like more insight and specific reasoning documented here)
• What parts of the module spec must be followed to the T in the node es module implementation and _why_, from a technical perspective? (JS engines have a long history of implementation spec nonconformance in some edge cases - usually for good reasons, such as compatibility, though I think most would prefer not to admit that) I don't think giving up on a large set of the end-user features we'd like to have a good transition or ecosystem experience because we've been told that it'd be nonconfoming in ways that are unlikely to matter _in practice_ is not a great reason to give up (IMO, this indicates a failing or the original spec to account for the node use-case particularly well, and may warrant revision of the spec by the tc39 in the future, once all the usecases are known and possibly there's even a nonconforming implementation to standardize on).
• What browser esm do we expect to be executable in node (without processing by a tool)? What node esm would we expect to be executable in the browser (without processing by a tool)? (There's no conditional imports in es6 modules, so all feature detection must happen via globals, and all imports must be unconditional - these are some big caveats!) I, personally, would expect something like lodash or another util library to be able to to be cross-consumed; but today there are a pile of constraints on that (mostly by the browser) - only relative paths, all extensions must be specified (and browsers/webservers are going to lag with adding support for detecting/serving up .mjs files as javascript, which is another challenge imposed on evolving the node ecosystem by browsers). Realistically, this means most code written in a node-first way will need preprocessing before being consumable by the browser. Additionally, any browser code that uses fully qualified URLs will need preprocessing before being consumed by node. In some ways, this is a _huge_ step backward from commonjs and amd module loaders, since you'll definitely need to at least _check_ your modules in some tool before consuming them on some platform. 😞

[devsnek]

i can answer Does module execution order have to be maintained compared to what the user wrote? with a strong yes. check out https://github.com/nodejs/node/pull/16675 near the end.

[weswigham]

@devsnek

i can answer Does module execution order have to be maintained compared to what the user wrote? with a strong yes. check out nodejs/node#16675 near the end.

I'm aware, but the notes from the tc39 meeting also say

DH: It feels like sometimes a spec is trailing reality, not leading it. We can just go away to implementations and after studying what the implementation details are we can spec it. I feel from the room we have a consensus of wanting something in spec but not knowing enough about the problem.

And

Conclusion/Resolution
This goes against the sort of spirit of the specification, though we have no particular normative spec text to add right now. Spec may in the future add normative text around this.

Which implies "that sounds nice, but we can't actually guarantee it (at least in simple terms), and it isn't actually true if you consider that imports are cached".

Additionally, that only matters within a specific module graph anyway - consider that if any import can be a dynamic import object using a loader hook, then theoretically any import could cause a new subgraph to be created (via the execution of a new vm context during resolution) which inherently has ordering unbound form the parent graph (though, do these share a cache like requires do? If they don't you get multiple execution instead of out-of-order execution, which is arguably worse - though you could make a global cache yourself... I suppose you'd have to in order to handle circularities). There's a lot of nice-to-haves there that can't _actually_ be guaranteed (at least not in the flexible, extendable world node would like to present).

And to top that off, for internal modules in https://github.com/nodejs/node/pull/18131 it was being discussed that that PR (internally) messes with the execution order by making sure that all the internal libs get executed before public packages so _they_ can get named imports.

So while the answer is yes, I definitely feel more like it's a qualified yes insofar as nothing surprising should happen, with surprising being defined a little bit differently for everyone involved...

[devsnek]

18131 is possible because we know internal modules won't error on evaluation, and that's why the other pr was turned down, it creates inherent instability in the module record if it fails like that.

[giltayar]

To add to @weswigham excellent list of questions, and to refer specifically to this part of his last point:

Realistically, this means most code written in a node-first way will need preprocessing before being consumable by the browser.

• Can we (and should we?) approach browser vendors and/or TC39 with a proposal that will enable them to implement additions to browse ESM so that the browser _can_ consume Node ES Modules without any preprocessing?
  
  
  
  • Obviously, I believe we can. I believe Node and the browsers can meet halfway, and if we come with a concrete proposal that is easily implementable, this would be to the benefit of both sides. The npm proposal for ES Modules has a hint of how this can be done, but we need to fully flesh this out.
  
  

[ljharb]

Because paths/URLs differ across environments, the only possible way I can think of to make that work is if browsers are willing to add loader hooks that would allow you to configure them to load modules "as if" it were node - or, if node users were willing to use URLs exclusively, and use loaders in node as needed. The latter, imo, is unacceptable.

TC39 has no bearing on the conversation unless browsers are first convinced.

[weswigham]

Can we (and should we?) approach browser vendors and/or TC39 with a proposal that will enable them to implement additions to browse ESM so that the browser can consume Node ES Modules without any preprocessing?

This is a very pessimistic view, but browser implementers have no concrete incentive to implement anything module-related to align with what node needs for an ergonomic es module implementation at present. There's no existing ecosystem for them to benefit from, and it will still cost implementers time and tech debt - essentially you're selling them on the future promise of the node es module ecosystem. They also can just choose to take a "wait and see" approach in that regard - they can see if node es modules don't flounder on delivery, then if they become wildly successful they can extend their minimal module support to support loading them. (They do have their own strange compatability issues to worry about) I'm not saying we shouldn't try, and I don't think we'd get a straight "no", but the incentives don't seem like they've been there in the past when es modules were being designed. Looking over notes from the past few years, almost every time someone has gone to the tc39 to ask for changes or permissiveness in the existing spec to standardize something node wanted as permissible, the answer has not been positive (save import.meta). That's part of what's driving everything towards the current spartan state of affairs. My observation is, perhaps, a little more bleak than is warranted, as I've only been an observer of these processes.

_But I think, based on the last year of tc39 notes, what it looks like the committee needs to see is node doing what it must to make es modules optimally usable for its ecosystem, at which point it can come back and create or update the standard around what's been done_. In fact, I can quote @wycats stating almost exactly that, from one of the links above from last year:

YK: approach, get as close to the spec as possible, and come back to the committee with concrete points that need to be fixed.

This has been a bit of a tangent. sorry. I've been watching this space for years, and the near zero apparent external progress here has just been flabbergasting given how much es modules have been adopted by transpiler-using-programmers since even before the es6 spec officially dropped - this feeling is probably what's helping drive this rush to implement _something_ before node 10 drops.

[MylesBorins]

As an aside... There is no need to rush anything before Node 10 drops. We can remove the flag prior to LTS if it stabilizes. It would also be possible to drop the flag in a semver minor release during active LTS. So the runway is more like 9 - 27 months

[WebReflection]

some answer for @weswigham hopefully interesting for others too.

the time should be taken to get it right

I couldn't agree more.

What, specifically, is going to make this feature more attractive than a transpilation step to commonjs, especially if running on multiple platforms is going to need a buildstep, anyway?

There is no need for a build step to run cross platform a self contained module.

Actually, there is no need for build steps for any ESM module (more later).

Build steps are rather a best practice for the Web, JavaScript is not a Web or Node only language.

• SpiderMonkey already has ESM (static, not dynamic)
• JSC already ships both
• GJS (GNOME JS bindings) will use SpiderMonkey mechanism
• Kinoma, Espruino and others might use ESM soon too

What browser esm do we expect to be executable in node (without processing by a tool)?

Not browser esm, any code that doesn't use NodeJS core modules could be shared.

What node esm would we expect to be executable in the browser (without processing by a tool)?

Not browser only, any code that does not use NodeJS core modules can land in npm and be used env-agnostic.

There's no conditional imports ... these are some big caveats!

Dynamic import(stuff ? that : other) partially solve this but I agree few will struggle with this caveat.

any browser code that uses fully qualified URLs will need preprocessing before being consumed by node

Two slightly off assumptions here:

• node will be incapable of resolving fully qualified URL in import
• fully qualified URLs are needed at all

CommonJS Example

Two modules, _acjs_ and _bcjs_, both installed, hence inside _node_modules_ folder.

acjs/index.js

const bcjs = require('../bcjs');
console.log(bcjs);

bcjs/index.js

module.exports = Math.random();

The execution of node -e 'require("acjs")' will log a random number

ESM Example

Two modules, _aesm_ and _besm_, both installed, hence inside _node_modules_ folder.

aesm/index.js

import random from '../besm/index.js';
// print via JSC or SpiderMonkey
print(random);

besm/index.js

export default Math.random();

The execution of js52 or jsc -m node_modules/aesm/index.js will log a random number.

Does it work on the Web ?

TL;DR Yes, if you use the same CDN to publish your modules you can actually import in the same relative way.

Example, instead of import ... from 'hyperhtml/esm' in this file which indeed requires bundling alchemies to work,
I could've written (and I will ASAP) import ... from '../../hyperhtml/esm/index.js' and that would've worked out of the box for development, where I would bundle anyway the module for production.

Also notice-able that specific project with Custom Elements runs in NodeJS too thanks to basicHTML, yet I need to convert ESM to CJS because there's no ESM in NodeJS (like you said .mjs is not a portable solution).

As summary

It would be great if we could stop thinking binary (Node / Browser) when the official JS module system can, and will, affect many other situations but also we should start considering that, as shown in these examples, cross portability is already reality.

Please let's get this right, thanks for reading.

[evanplaice]

For bare imports

@giltayar

Can we (and should we?) approach browser vendors and/or TC39 with a proposal that will enable them to implement additions to browse ESM so that the browser can consume Node ES Modules without any preprocessing?

Browser vendors, no. TC39, yes. If it goes through TC39, the functionality eventually land in browsers. The downside being, TC39 is slow so addressing this sooner than later will be best.

Also keep in mind the potential for npm asset for front-end specific modules. Depending on how that pans out 'in the wild' it may not even be necessary to provide bare imports to node_modules.

Are we using import.meta?

If so, keep in mind that the spec is still only Stage 3. If we're going to use a non-finalized spec we should reach out to TC39 to identify any potential footguns ahead of time.

[ljharb]

@evanplaice I think you misunderstand how TC39 works; browsers are members of it, and have actively blocked some of the changes that have been requested for purposes of the node integration. It's not that TC39 is "slow", it's that it has many stakeholders - appropriately many - and so it's often harder to ship bad decisions (but not impossible, obviously).

[evanplaice]

@ljharb Copy. Scratch that last comment.

[linclark]

I can likely do a code cartoon on the ESM spec for this. I also may be able to present on the tricky issues for making WASM modules work as ES modules.

[bmeck]

Scheduling the Summit

I am thinking we should do this at the end of March or early April. It hopefully has enough lead time for an online presence to be possible for members, but is not too far out to be irrelevant to current discussion. To that purpose I have made a doodle for times dates seem reasonable.

• Fill out the Doodle

Since I have been assigned this issue, I am also wanting to see if there is any time limitations for members. This will be a long meeting and we have a geographically diverse set of members. I expect planning to be closer to UTC than PT. I think we can leave the doodle open for a few weeks and people can fill it out or voice ideas about this here.

BE SURE TO CONSIDER THE SCHEDULING OF TOPICS BELOW WHEN CHOOSING DATES

Topics Presented at the Summit

This is a rough plan for how we are going to handle our large group but limited time for the summit. I am open to altering the nature of presenting and deciding upon topics as members see fit.

We have a rough idea of topics for this it seems. Our large group of active members and limited time for the summit has makes me want to aggregate topics separately from deciding on the presentation of those topics. This has led me to create a form to aggregate topics so interested parties can prepare presentations or discussions on specific topics of interest without a single member being the only voice heard on a specific topic during the summit. Two weeks prior to the summit, the form will stop and topics to be presented will be decided. Based upon responses to topics and the nature of topics submitted I will decide upon the agenda and publish it. Members can agree to the agenda at that point. Members presenting these topics and times of discussion may vary even after publication of the agenda. The time allowed for discussion of a specific topic should not change once the agenda is agreed upon unless the presenters are unable to attend.

• Form to Submit a Topic

[bmeck]

People might have missed this scheduling doodle since it is a comment, we are still missing replies from:

@benjamn
@chrisdickinson
@dduleone
@WebReflection
@refack
@tbranyen
@eugeneo
@evanplaice
@jasnell
@bmeurer
@iarna
@Fishrock123
@XadillaX
@abouthiroppy
@mhdawson
@zackschuster
@devamaz

[tbranyen]

@bmeck don't see my name listed, is my answer from the previous doodle being counted or am I put on an observer status now?

[bmeck]

@tbranyen mistake on my part. I was going through the list from https://github.com/orgs/nodejs/teams/modules/members?utf8=%E2%9C%93&query=+membership%3Achild-team and manually adding things. Teams are a sad experience on github right now with few features :p. My apologies.

[MylesBorins]

@tbranyen I believe that @bmeck took the list in the current readme which was created prior to you accepting an invitation to the org. You can see your name being added here

[bmeck]

@nodejs/modules

We are still missing responses from 22 out of our 41 members.

[benjamingr]

We can discuss this in the agenda and remind people to fill out the doodles. @bmeck

[manekinekko]

@bmeck sorry for the delay. I responded to the doodle.

[mduleone]

Sorry for missing the doodle! I just responded.

[bmeck]

I will be closing the doodle tomorrow. There are 3 dates that are tied, but I am leaning on Tuesday April 3rd if the tie remains after closing.

[tbranyen]

@bmeck added myself to the doodle, sorry about that!

[Fishrock123]

I assume it's closed - April 3 works fine for me.

[MylesBorins]

I've updated the top comment to make it easier to find the date / proposal link.

submit proposals!!! --> https://docs.google.com/forms/d/e/1FAIpQLSd4tQLH-8ajvDHEqJoJXjEftU8gSliVMCvUoasbAS8CUK8lDQ/viewform

[bmeck]

Things have been busy but I have a preliminary schedule for things up at

https://docs.google.com/document/d/1oAvF_qukb1dIUWji9GjHNCL5-0uJ3_M-yPG-lzJVBbs/edit#

I'm emailing topic leaders to coordinate, but I suspect we will have slides for relevant topics but will also have a lot of discussion and need to be sure we pad time so that people can discuss things and add questions.

Since the summit is so long and we have international members, I would suggest we plan for early morning Pacific as I said earlier. 7AM PT would mean that the summit would end at midnight in Germany and we even have members further east.

Is there any major complaint with waking up that early?

[linclark]

I posted a code cartoon on ES modules which might be good preparatory background reading for this (and also posted the video of the WebAssembly presentation that I gave at the last meeting).

[jdalton]

Just to confirm the summit is April 3rd (so this coming Tues)

[bmeck]

Yup.

[devsnek]

do we have a confirmed time yet? i've seen maybe 7AM PT so far but knowing for sure would be nice so i can get excused from my classes 😓

[bmeck]

@devsnek yup 7AM PT.

[MylesBorins]

@bmeck I may have to duck out part way through the day. Would I be able to be moved a bit sooner?

[MylesBorins]

also... how are we planning to broadcast? Hangouts on Air? Zoom? Let me know so I can help set it up

[benjamingr]

It starts at 5pm my time and it's holiday here (Passover) - I will do my best to attend as much as possible!

[jdalton]

Just a FYI, if you're looking for folks to move or adjust for @MylesBorins I'm totally flexible. I've blocked off the whole day so I'm free whenever is needed.

[bmeck]

@jdalton @MylesBorins I have moved you both around as well as myself, I wanted @jdalton 's talk to be available to most and tried to keep it from being at the end of the summit.

[jkrems]

@MylesBorins Can we update the top post with the time, maybe using the standard snippet?

Time

UTC Tue 03-Apr-2018 14:00 (02:00 PM):

| Timezone | Date/Time |
|---------------|-----------------------|
| US / Pacific | Tue 03-Apr-2018 07:00 (07:00 AM) |
| US / Mountain | Tue 03-Apr-2018 08:00 (08:00 AM) |
| US / Central | Tue 03-Apr-2018 09:00 (09:00 AM) |
| US / Eastern | Tue 03-Apr-2018 10:00 (10:00 AM) |
| London | Tue 03-Apr-2018 15:00 (03:00 PM) |
| Amsterdam | Tue 03-Apr-2018 16:00 (04:00 PM) |
| Moscow | Tue 03-Apr-2018 17:00 (05:00 PM) |
| Chennai | Tue 03-Apr-2018 19:30 (07:30 PM) |
| Hangzhou | Tue 03-Apr-2018 22:00 (10:00 PM) |
| Tokyo | Tue 03-Apr-2018 23:00 (11:00 PM) |
| Sydney | Wed 04-Apr-2018 00:00 (00:00 AM) |

Or in your local time:

• http://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Module+Summit+2018-04-03&iso=20180403T14
• or http://www.wolframalpha.com/input/?i=02PM+UTC%2C+Apr+03%2C+2018+in+local+time

Edit: Manually extracted from a diagnostics issue and edited. No warranty for the correctness of my editing job.

[MylesBorins]

@jkrems updated

[linclark]

Unfortunately I have other meetings from 11:30 Pacific to 12:30 Pacific. Could I possibly switch with Bradley or Justin?

[bmeck]

@linclark swapping with my slot

[SMotaal]

Sorry if this is not the appropriate place to post, but is there a way for non-members to watch a live stream (or something to that effect).

[justinfagnani]

@bmeck @linclark Sorry, I see that I have 12pm and 2pm PT meetings. I think swapping with where Lin is now would help out there. Lin can you do that? Thanks.

[GeoffreyBooth]

Is there a link to join or watch the meeting?

[MylesBorins]

I just updated the OP.

We will be streaming to youtube, and I'll place a link when we are live. We had not discussed making the panel open to non-member so I think it makes sense to assume that the zoom link is currently only for Team Members

[MylesBorins]

@bmeck I'm not sure I have an hour worth of content for tomorrow. Feel free to shorten my time to 30 minutes if it helps with juggling folks.

[bmeck]

Members, let us know if you have trouble connecting to the Zoom link up top.

[bmeck]

Stream starting at 7:15AM PT

[MylesBorins]

People following from home can stream at http://www.youtube.com/watch?v=O-7rXALxL9Y

[benjamingr]

I'm having connectivity issues, I'm on the YouTube stream in the moment and am going to drive home to try to attend with Zoom.

[MylesBorins]

Looks like someone else just started a meeting on the same zoom account. Give me 10 minutes

[MylesBorins]

Due to technical difficulties we are moving to hangouts on air

participants please check out https://hangouts.google.com/hangouts/_/tmdgbvvtyraerlk4twtlqqejfqe

A new stream link will be available v soon

[benjamn]

[MylesBorins]

Ok finding another options :sigh:

[MylesBorins]

Original Webinar is working again

https://zoom.us/j/486767135

I've pinged foundation to see if we can avoid getting booted

[mhdawson]

Can other people get in? It just says "If you are the host, please sign in to start the webinar"

[bmeck]

@mhdawson I got in

[linclark]

@justinfagnani I can switch times with you

[MylesBorins]

Great work today everyone! All the videos are available individually on youtube now, shared in a larger play list

https://www.youtube.com/watch?v=IbPxOaGveXA&list=PLfMzBWSH11xb9LoHLO0q3Yo7HXjMOm11N

[GeoffreyBooth]

Can you please post links to the various presentations that were shown today? Not the video, but the Google Docs/Slides/etc. documents that people presented. Thanks!

[MylesBorins]

I'll cross post over on youtube when people post stuff here.

@GeoffreyBooth if this gets lost in the noise I'll start asking folks for the links. Please feel free to ping again if this falls off my radar

[jdalton]

@MylesBorins

Slides: https://goo.gl/WLufhg
Twitter: @jdalton
Repo: standard-things/esm
Reading:

• Release blog post
  
  
  
  • ES modules: A cartoon deep-dive
  
  
  • Top-level await proposal: Variant C