Build-image: Use current supported language versions by default in Focal branch

[verythorough]

I noticed that some of the default language versions in Focal's included software list are EOL. I think it's great to provide ways for users to run/install older versions to help ease migration from Trusty and Xenial, but it would be great to get new sites started with current supported versions.

Here are a few that I've noted:

• Ruby 2.6 is still supported, but our default version is 2.6.2, while the latest patch is 2.6.7. Is it possible/advisable to pin Ruby to minor versions instead of a specific patch?
• Node 10 reached full EOL in April. Current supported versions are 14 and 16. Active support for 14 ends in October, but 16 was just released in April. I'm thinking 14 is probably safer for now, but folks in the dev tooling guild would have more expertise.
• Python 2.7 reached EOL in Jan 2020. The other version we have on Focal is 3.8, which will be good all the way until Oct 2024! It seems like a good version to default to, and we'll still have 2.7 as an option for migration support.
• Go 1.12 hit EOL in Feb 2019. Current supported versions are 1.15 and 1.16. Gophers on the team might have thoughts about which version makes most sense here. To be honest, I'm not sure how much people actually use the Go runtime in builds, anyway, since they often use compiled binaries (as with Hugo, for example).
• Elixir 1.9 is not the current version, but it does still get security patches, so maybe we just keep this one for the sake of stability, and consider upgrading if we get an issue for it.

In some cases, it may be that the included software list is just out of sync with the actual behavior, in which case we can just update the doc. In others, I think we'll want to update the build script, too. Let me know if I'm missing something!

[ingride]

Imma reassign this to myself when i actually start working on it, just in case someone else might wanna do it

[Benaiah]

@ingride @verythorough how do you feel about splitting this into separate issues, one for each of the dependencies?

[ingride]

I don't mind, you might wanna do them all at once tho because the build image takes so long to build, test, and release that multiple PRs are actually a disadvantage here.

[ingride]

Ruby 2.6 is still supported, but our default version is 2.6.2, while the latest patch is 2.6.7. Is it possible/advisable to pin Ruby to minor versions instead of a specific patch?

In the Dockerfile we specify that we kept 2.6.2 because many customers are pinned to it. We should check this is still true. But for the purpose of this, which is new sites before defaulting for everyone, we can prolly move to a patch, and it should be ok for people to move from 2.6.2 to this version by changing their env vars. But we might wanna keep a 2.6.x along with 2.7.x rather than removing 2.6.x altogether.

Go 1.12 hit EOL in Feb 2019.

We actually have Go 1.14.4 https://github.com/netlify/build-image/blob/focal/Dockerfile#L390 . We'd probably be ok going to latest.

Python 2.7

Python is hard to migrate from 2.x to 3.x but maybe that's ok that we don't support it anymore.

[verythorough]

To be clear, my priority with this issue is for the _defaults_, which only apply to new sites, and can be overridden with env vars, etc. if other versions are available.

As for whether we keep older versions available for easier migration of existing sites, I'm cool with y'all making the cost-benefit analysis of weighing that against the risk of a sudden upstream support change upending our ability to deploy the image.

[Benaiah]

In the Dockerfile we specify that we kept 2.6.2 because many customers are pinned to it. We should check this is still true. But for the purpose of this, which is new sites before defaulting for everyone, we can prolly move to a patch, and it should be ok for people to move from 2.6.2 to this version by changing their env vars. But we might wanna keep a 2.6.x along with 2.7.x rather than removing 2.6.x altogether.

@ingride are you suggesting we install all of 2.6.2, 2.6.x, and 2.7.x in the Dockerfile? This makes sense to me because of the number of sites pinned to 2.6.2 which would get an upgrade path to 2.6.x that won't slow down their builds. What do you think about also installing 3.0.x? That seems unnecessary to me but perhaps we'd want it preinstalled to encourage people to use it if they can.

the latest [Ruby] patch is 2.6.7

For some reason, the latest version that is seen by rvm is 2.6.6. I'm not sure why this is, because 2.6.7 was released on April 5. See rvm list known and rvm strings 2.6. If we decide to start defaulting to 2.6.x instead of a particular patch version this shouldn't be an issue.

Regarding Node, I think we should install and default to 14.17.x. Node 14 is the current LTS release and does not reach EOL until 2023-04-30. We may want to also install 16.3.x, since 16 will become the LTS release in October. The best-supported solution would be to default to 14.x.x but I worry that this presents a small risk of builds breaking without changes to the build or pinned versions.

Regarding Python, I agree that we should start defaulting to 3. If we're already changing the defaults, I feel like we should consider using 3.9.x, which is supported thru October 2025.

Regarding Go, I think we should update the default to 1.16.x, the newest version. Go is remarkably backwards compatible and, as @verythorough pointed out, most people using Go in their build aren't even going to use the Go compiler that's installed by us, since things like Hugo typically come as a pre-compiled binary. EDIT: the primary use of our installed Go runtime is likely to be building people's Go functions, though this is a pretty small set of users compared to those using JS functions.

I don't have any particular preferences on Elixir support. We're not actually pinning the Elixir version in any way or supporting the user pinning it - all we do is install esl-erlang (this is the "complete installation" of erlang) and elixir using apt-get. For example, when building the build-image locally just now, the version of Elixir I ended up with was 1.12.0 using Erlang/OTP 24 and erts 12.0.2. I would be fine with continuing the current behavior of using the Erlang and Elixir versions that happened to be available thru apt-get at the time the build image was last built. I suspect the number of users using Elixir is pretty low, or we would likely have had someone complaining about a build breaking due to the unpinned version by now. I'll update the list of included software to note that the version of Elixir contained is whatever was in apt-get when the build-image was built.

This might be more work than we want to do right now, but I imagine it could be somewhat useful to replace the included_software.md file with a site that calls the various pre-installed language runtimes to get the actual current versions of all these languages in the build-image. This would prevent the list of included software from getting out of sync with the build image or out of date.

@ingride @vbrown608 @ehmicky do you have any thoughts on my suggestions here?

[verythorough]

Regarding Node, ... The best-supported solution would be to default to 14.x.x but I worry that this presents a small risk of builds breaking without changes to the build or pinned versions.

We pinned/installed Node at the major level for most of Netlify's history, only switching to a specific patch version 14 months ago. I can't find the reasoning for switching to a specific patch version, but it seems that based on our previous history, the failure risk is low? Curious what folks like @ehmicky think about this.

This might be more work than we want to do right now, but I imagine it could be somewhat useful to replace the included_software.md file with a site that calls the various pre-installed language runtimes to get the _actual_ current versions of all these languages in the build-image. This would prevent the list of included software from getting out of sync with the build image or out of date.

That would be cool indeed! Every time I look at those files I find a new discrepancy. Agreed that it might not be worth the effort right now, though. For a short-term/low-effort alternative, perhaps we add something about updating that file to the PR template?

[ehmicky]

We pinned/installed Node at the major level for most of Netlify's history, only switching to a specific patch version 14 months ago. I can't find the reasoning for switching to a specific patch version, but it seems that based on our previous history, the failure risk is low? Curious what folks like @ehmicky think about this.

Yes, I think the risk of not pinning the minor version is very low compared to the benefits of it. Node.js is pretty good about semver. Also, bug fixes and security fixes might come in a later minor version of the current major release, so pinning the minor version might prevent from getting those. Based on this, in my opinion, we might prefer defaulting to 14.x.x instead of 14.17.x.

[ingride]

@Benaiah, by 2.6.x I meant only having one 2.6 version, not all of them, but definitely at least one, along with the 2.7 versions. I think 3.0 could be nice to have, but could also be done at a later stage.

No strong opinion on 2.6.6 vs 2.6.7, theoretically rvm knows how to install from source even if the binary is not available - I did that locally, but it's probably too slow to consider, so 2.6.6 sounds good to me.

rvm install 2.6.7
Searching for binary rubies, this might take some time.
No binary rubies available for: osx/11.2/x86_64/ruby-2.6.7.
Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.
Checking requirements for osx.
Installing requirements for osx.
Updating system......
Installing required packages: autoconf, automake, libtool, pkg-config, coreutils, libksba, zlib............
Certificates bundle '/usr/local/etc/[email protected]/cert.pem' is already up to date.
Requirements installation successful.
Installing Ruby from source to: /Users/ingride/.rvm/rubies/ruby-2.6.7, this may take a while depending on your cpu(s)...
ruby-2.6.7 - #downloading ruby-2.6.7, this may take a while depending on your connection...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.4M  100 13.4M    0     0  28.5M      0 --:--:-- --:--:-- --:--:-- 28.4M
ruby-2.6.7 - #extracting ruby-2.6.7 to /Users/ingride/.rvm/src/ruby-2.6.7.....
ruby-2.6.7 - #configuring.............

[ingride]

@Benaiah is there a draft PR for the changes?

[Benaiah]

@ingride I'll have one up later today.

[Benaiah]

I have a draft PR up at https://github.com/netlify/build-image/pull/588 now. To summarize the changes:

• Python 3: updates default from 3.8 (which is the default present in the system before installing packages) to 3.9, and uses 3.9 to install pipenv (instead of 2.7).
• Ruby: updates default from 2.6 to 2.7, and additionally installs 3.0.
• Go: updates default from 1.12 to 1.16.
• Node: updates default from 12.18 to 16.4 (suggested by @ehmicky).
• NVM: updates from 0.35 to 0.38.
• Swift: updates default from 5.2 to 5.4.
• Hugo: updates from 0.82 to 0.85.

[Benaiah]

I'm going to investigate what will happen if we specify some of these versions at a level higher than the patch version. The existing PR pins to the current patches for each version instead of not specifying them - I don't think we reached a conclusion on which we want to do moving forward.