Build: Build plugins do not work on Node 6 (or earlier)

[ehmicky]

Build commands and Build plugins are spawned in child processes. Those child processes uses whatever Node.js version the user has chosen (for example with .nvmrc) (default: Node 10 on Xenial).

However, in order to run Build plugins, we wrap them with some instrumentation code (the files in that folder). That code is responsible (among other things) for:

• Calling the event handlers in order
• Handling errors
• Communicating information back/from the parent process
• Loading the plugin
• Validating the plugin shape
• Normalizing the plugin shape (e.g. backward compatibility fixes)

The problem is that this instrumentation code has to have a minimum Node.js version:

• It uses some dependencies with minimum supported Node.js version
• It uses core Node.js methods introduced at some point in Node.js release history

At the moment, that wrapper code has a minimum Node.js version of 8.3.0, which means Build plugins won't work if the users Node.js version is below.

What we can do to mitigate the problem:

• Try to move as much logic as possible from the instrumentation code to the parent process (which uses Node 12 and is decoupled from user's version). Not everything can be moved though.
• Remove or downgrade some dependencies. This might be hard for some dependencies such as our own Netlify JavaScript client.

Another approach would be to enforce Build plugins to declare their minimum supported Node.js version (for example in engines.node in package.json). Users would need to upgrade their Node.js version to use specific Build plugins.

---

Side note: we also have some core Build plugins which are always loaded (one for caching and another for bundling Netlify Functions with zip-it-and-ship-it). So every Netlify Build user will use those plugins. So, even without any Build plugin, builds of users with old Node.js versions would crash. However this can be solved by either:

• Removing the concept the core Build plugins and moving their logic to the core codebase instead.
• Always loading those core plugins with the parent process Node.js version (Node 12) instead of the user's Node.js version

Data showing builds (within last hour) Node.js version.

[fool]

MVP suggestion: we somehow cause build plugins to error when run with the too-low node version with a good error message that node version upgrade will be needed. THE WHOLE BUILD SHOULD FAIL!

Existing (PUBLIC) issue requesting support of the engines flag in package.json. People don't all agree on how to use it though so this is somewhat fraught with peril: https://github.com/netlify/build-image/issues/53

[ehmicky]

What you are suggested was suggested by @DavidWells as well here. I agree we might need to do this in the short term.

However, at some point, we will make Netlify Build public to every user, so we need a strategy for this to happen without breaking users on Node 6.

For the engines.node flag, I think the issue you are linking is about using it like we currently use .nvmrc. What I was suggesting was instead for Build plugins (not user repositories) to declare their minimum Node.js version (with engines.node or .nvmrc) and print an error message if a user is trying to use a Build plugin whose minimal Node.js version does not match theirs.

To fix the above issue, this would need to be coupled with an enforcement of the minimal Node.js version any Build plugin can have.

Any other solution that does not introduce breaking changes is welcome too :+1:

[fool]

oh, thanks! I didn't understand that scope, but I think you are on the right track with not breaking legacy users' builds :)

Here's query to help quantify how many people are using what node versions in prod today: https://cloud.humio.com/netlify-production/search?query=%24bbot()%20%7C%20%2FNow%20using%20node%20v(%3F%3Cver%3E%5B0-9%5D)%5C.(%3F%3Csubver%3E%5B0-9%5D)%2F%20%7Cver%20!%3D%20%2F%5E(10%7C12%7C13%7C11%7C9)%2F%20%7Ctop(ver)&live=false&start=1h&fullscreen=false

[ehmicky]

I just filed #1173 to fix the issue with core plugins (which is only half of this issue).

[ehmicky]

I have submitted #1181 which should fix the issue with core plugins. So Node 6 builds should not fail anymore when no Build plugin is used. However when a Build plugin is used, they should still fail because of the reasons mentioned above. So we still need a strategy for supporting those versions. @verythorough @erquhart

[verythorough]

Working on a discussion issue for that today!

[calavera]

However, at some point, we will make Netlify Build public to every user, so we need a strategy for this to happen without breaking users on Node 6.

I don't agree with this, and it will suck for some customers. We cannot maintain runtimes that are not maintained by the language maintainers anymore. I rather make the work to communicate to customers still using Node 6 that we don't support it anymore. We should give them two options:

1. They upgrade to a newer supported version before X date.
2. We upgrade them automatically if they are still deploying with Node 6 after X date.

I know @verythorough started a document about runtime release versions, and we should implement the work to keep our customers up to date before trying to maintain all runtime versions forever.

[ehmicky]

This sounds good!
The one thing to keep in mind is that X date is currently GA date when it comes to Node.js 6 users that want to use Build plugins. It just does not work on Node.js 6 due to the reasons explained above.

[mheffner]

@ehmicky How many build plugins only support Node.js 6?

[ehmicky]

None. Most Build plugins currently support Node.js >= 8 or >= 10.

The problem would be that Node.js 6 users:

• Currently get a weird-looking exception error making their build failed when they opt-in to the Build beta. This could be more of an issue when the GA is out. This hopefully should be fixed by implementing this.
• Cannot use Build plugins without getting a similar weird-looking exception error.

Following up from what @calavera was saying about deprecating Node.js 6 at the company level. Until this deprecation takes effect, if the user is using Build plugins (e.g. if @netlify/config returns a list of plugins), should we print a warning message telling them that Node.js 6 is not supported with Build plugins? We could then either make the build fail, or not use any Build plugins with that build.

[verythorough]

Do we know how many site on Xenial are using Node 6? I suspect it's very few.

In any case, I appreciate that #1239 will (hopefully) keep Node 6 users' builds from breaking when we flip over to Netlify Build for all Xenial sites, because we don't want builds to break without user action.

On the other hand, installing a plugin definitely counts as a user action, so I'm good with a build-failing error, preferably one which bubbles up into the deploy summary. (If it could take advantage of the existing plugin error functionality, that would be ideal.)

[ehmicky]

So it seems the action points would be:

• We need to implement this and ensure Node.js 6 builds on Xenial (which might not be lots of users) are working
• We want to deprecate Node.js 6 at the company level, which is tracked as part of https://github.com/netlify/engineering/issues/50
• We want to print a warning message and make the build fail for Node 6 users that use Build plugins (tracked by https://github.com/netlify/build/issues/1278)

Please let me know if I got this wrong.

[verythorough]

Sounds good to me!

[ehmicky]

The first point is fixed by #1299.
The third point is fixed by #1304.
So all there is left is tracked by https://github.com/netlify/engineering/issues/50.
Therefore I think this issue can be closed.