Shields: Upgrade Heroku stack from heroku-16 to herkou-18

[paulmelnikow]

We're using an out-of-date stack in all our Heroku apps. We should review the changelog, test out the latest version, and get ourselves up to date.

There's a button in the UI to do the upgrade. We could also try it on a review app first, before doing it on staging.

[PyvesB]

The official announcement contains little information, apart a link to the main docs and stating that it is based on Ubuntu 18.04.

This page contains a detailed table with the versions of all the different packages on the image.

I've got a feeling that we should just go ahead and give this a try. However, I'm not sure we can just upgrade the review apps in isolation, given that these are based on the parent shields-staging app.

[paulmelnikow]

Agreed. I'm not sure either, though I think it might be possible to upgrade a review app that has been created for an existing PR.

[PyvesB]

https://shields-staging-pr-4968.herokuapp.com/ will be upgraded on next deployment, i.e. commit on that PR.

[PyvesB]

Everything seems to be running happily on that particular review app, therefore I'm about to upgrade the staging app which will in turn impact all new review app deployments.

I suggest we leave the resulting state of affairs in place for a few days. If we don't notice anything odd on the various review apps and the staging environment, we can then upgrade one of the production regions first (if we have the EU one setup properly by then), and gain confidence that it runs smoothly in real conditions before upgrading the second one.

For reference, a rollback of stack versions can easily be performed via the CLI, see the instructions here.

[PyvesB]

Nothing unexpected so far. Will aim to upgrade the production environment tomorrow morning European time.

[PyvesB]

Well, opening the Heroku console this morning, it's already done on the production environments... Slightly worrying that big changes like this go through without anyone being aware and with no traces in the activity log.

[paulmelnikow]

Great!

That would be a good thing to have in the logs, for sure. Fortunately it happens infrequently, and usually seamlessly.

[chris48s]

Slightly worrying that big changes like this go through without anyone being aware

Is it possible someone promoted the staging instance to production after you deployed the new build pack to staging and didn't realise?

[paulmelnikow]

Ah, that's a good point. Flipping the buildpack switch in production doesn't in practice have any effect, since the builds are production builds are run on the staging app (and then promoted).

(Oops, that's probably wrong. The stack includes more than just the buildpack.)