Renovate: Renovate package.json > engines fields

[rarkins]

e.g. keep node and npm versions updated

[Urigo]

I was wondering about engines support mostly for the use case of node version in apps.

I we discussed on email, there are a few questions to think about.
For example - updating to a major release might be too much.
if someone uses Node 4, maybe they want to get just one PR suggesting to update to Node 8.
If they reject it, they would probably wouldn't want to keep getting updates every time there is a new minor or patch version to Node 8.

Some possible solutions:

1. Take into account less than on engines, for example:

{ "engines" : { "node" : ">=4.10.3 <8" } }

Here we can keep getting updates on Node 4, 6 and 7 versions, but not on 8.

1. Some configurations on renovate.json
2. Internal logic to send just one PR for major version (basically just notifying that there is a new major version).

My personal opinion is that solution 1 is the best

[tunnckoCore]

I can't get the "idea" of this "feature". Why would one to update engines versions from time to time? I will have node >= 6, npm >= 5 and yarn >= 1 for all packages soon and i don't care what new version is out..

[Urigo]

@charlike one reason I can think of is that someone who uses babel or Typescript on the server, and when a new version of Node comes that supports ECMASCRIPT feature natively, he can ensure everyone uses that release and remove transpiling for those features from the code.

Another is to make sure specific bug fixes from Node are being patches in all the instances of that app

[rarkins]

Is this the best place to get node engines list from? https://nodejs.org/en/download/releases/

[Urigo]

@rarkins looks like it but I'm not sure.
This might be also a good idea? https://github.com/nodejs/node/releases

[rarkins]

I'm interested in this feature but need some "real life" examples. Perhaps it only works if you somehow "pin" the engines, e.g. <8 so that then you get an alert when 8 is released or 10.

[Urigo]

@rarkins I think there are few "real life" examples.

1. The server itself is dependent on features from that Engine, for example, using async/await without compilation.
2. Get important updates to your server, for example the latest security update that was released a few weeks ago.

People could use the engines field in their deployment scripts. For example referencing that in the nvm script

[Urigo]

I mean, I do want to get alerted when a new 8 is coming out.
but also when 10 is coming out....

[rarkins]

Sorry, I was unclear. I was thinking of "before and after" examples rather than so much why. I'm not sure if Renovate can necessarily "guess" what you'd want the semver updates to (eg simply add v8 or also drop v4?) but maybe most of the value would be alerting you that there's a new version available so you can decide what to do with it.

[rarkins]

This has also been requested by @vvo on Twitter: https://twitter.com/renovateapp/status/938004500798992384

He points to a package.json engines value of "node": "^9.2.0":
https://github.com/algolia/community-project-boilerplate/blob/ce7f140a807f329eec2370d501098c25be670a6f/package.json#L23-L24

I guess their policy is to always upgrade to the very latest version? 9.2.0, and next 9.2.1 or 9.3.0 I suppose?

[vvo]

I guess their policy is to always upgrade to the very latest version? 9.2.0, and next 9.2.1 or 9.3.0 I suppose?

Yes that's how we do it, I think about the nodejs version of a project setup the same way I think of dependencies: it needs to be as precise as possible to allow for env reproducibility. Since nvm is very easy to use, we have an nvmrc field in every project then it just uses the value inside as a nodejs version.

[tunnckoCore]

@vvo i was thinking about that strategy too. Sounds very good.

[ghost]

Some applications that I work with are configured with one of the following values for their node engine property:

• ^8.9.2
• ^6.12.1
• ^4.8.6

Only LTS versions of Node are used.

With this month's Node vulnerability advisory, we proceeded to upgrade the version strings to one of:

• ^8.9.3
• ^6.12.2
• ^4.8.7

By bumping the lower bound of the semantic version range, committing that change, and pushing the change to the application's repository, those application are re-deployed with a secure version of Node. (A good example of this is Heroku.)

It's a tedious, time-consuming, process to update the node property in each application.

[rarkins]

@destroyerofbuilds so would you be happy to update every time there’s a patch, or only when there’s a security patch?

[ghost]

so would you be happy to update every time there’s a patch, or only when there’s a security patch?

I can see value in updating on every patch:

• Our applications uniformly consume performance enhancements introduced in Node.
• Our application development teams have uniform access to the same functionality (resulting from the introduction of enhancements, or bug fixes).

Those lead to a consistent improvement to our applications, and ensures a high degree of consistency between application runtime environments (which improves developer productivity when working across applications).

My example above merely highlights that raising the minimum version is a requirement, regardless of cost, for security reasons.

If the process is automated, and the node property updated on every Node release, we gain the aforementioned benefits.

One counter argument against updating on non-security releases is the increase _noise_ generated by additional pull requests.

However, a counter to that counter argument is that the version bump is not a major version, and so it could be automatically accepted assuming CI passes.

[rarkins]

@destroyerofbuilds thanks for the feedback. I think the easiest possible first step would be one where we pin to LTS (8.x.x) or current (9.x.x) and bump that every time there's a node.js release. So it's good if you think that fits in with your workflow.

[ghost]

I think the easiest possible first step would be one where we pin to LTS (8.x.x) or current (9.x.x) and bump that every time there's a node.js release.

I believe that's fine.

The only drawback is that when Node 10 goes LTS next October, every project will receive a major version bump. Some teams may feel upgrading major Node versions to be a significant burden and elect to close the pull request.

I'm not sure what would happen in that case.

Will Renovate follow the existing major version workflow where it will accept the closed pull request as an indication that the team does not want to upgrade to that Node version, but still continue to auto-merge in patch upgrades for the current version of Node?

[rarkins]

Will Renovate follow the existing major version workflow where it will accept the closed pull request as an indication that the team does not want to upgrade to that Node version, but still continue to auto-merge in patch upgrades for the current version of Node?

Yes, with a little bit of work that functionality/behaviour can be ported over from npm to node

[hutson]

Coping a few comments I made in the Gitter channel to here.

---

A before/after example is here - https://github.com/renovateapp/renovate/issues/320#issuecomment-351767827

The benefits are documented in - https://github.com/renovateapp/renovate/issues/320#issuecomment-351767827

Those benefits are based on what is gained by Heroku-deployed Node services.

I don't see the purpose of always bumping to the latest but retaining the caret range

For an application there is no benefit. Pinning is fine.

Further thought is required for what to do for Libraries.

I think Renovate's existing workflow of pinning and upgrading minor and patch, alongside major upgrades as a separate PR, works well with the example outlined in the aforementioned comments.

So right now, I would like if Renovate was automatically submitting PRs to keep bumping the patch and minor versions of Node 8 on my Node service, and in October I would expect a PR that bumps to the new major LTS, Node 10.

Since I already have automerge configured for minor and patch, I would just expect Renovate PRs to be merged into the master branch automatically on Node 8 releases

Again, for a Node service.

Node libraries would need to have the widest possible allowable range to prevent Yarn from failing on installs at the library consumer level because of Node version mismatches

[hutson]

Further comments I've made in Gitter.

---

It would be possible that Renovate submits a PR to update the Node engine version, but before I accept that change, I accept a change to some code that still causes Heroku to re-deploy the service using the newer version of Node because we're using a carrot.

However, a better question is, do I want that?

I will still accept the Renovate PR, so my service would be upgraded shortly anyways.

Furthermore, the fact that the Node version for the service is upgraded for a change that was code only (and not related to the project's metadata) does not reflect my desire for a reproducable build.

Lastly, assuming I'm using Heroku, it's likely I'm using Heroku CI, and that a Renovate PR would not be auto-accepted unless Heroku CI passed, which would reflect whether the Node version upgrade works for my service.

So that workflow ensures a high degree of confidence that I can allow Renovate to automerge.

so if I implement this features for engines.node only, and for pinned versions - that works for you?

I wish there was a more concrete way to see if the project is an application or Node service.

I would like to avoid false positive matches against Node libraries

Furthermore, if it were possible to detect an application or service, then Renovate could submit a pin PR for the Node version

Similar to what it does for dependencies and development dependencies.

[hutson]

One downside that has come to mind is the fact that every developer on an application possessing a fixed Node version must all be using that version of Node.

Any attempt to execute yarn install with a Node in $PATH different than the version specified in the application's engines property will lead Yarn to fail.

For large single-application teams, it could be costly, and disruptive, to execute a node engine upgrade across the entire development team.

Furthermore, developers may contribute to more than one application, where each application is on a different fixed node version.

In the latter case mitigation is possible through the use of a Node version manager such as nvm.

[rarkins]

@hbetts anything that husky could do? e.g. a script when you switch branches or pull it checks if nvm needs to be run

[rarkins]

Also, isn't the flip side of this discussion that, "Developers on the same team/project are running it with different versions of node"?

[hutson]

anything that husky could do

husky combined with nvm to run nvm install as part branch/pull operations would work.

Also, isn't the flip side of this discussion that, "Developers on the same team/project are running it with different versions of node"?

Yes, that is _very_ common.

[rarkins]

You could also add —ignore-engines to true in a .yarnrc in any project where you don’t care about exact node versions. Or publish libraries sans “engines” fields.

[rarkins]

This has now been partially implemented - node engines with exact ("pinned") version numbers are upgraded. Further work awaits receiving further before/after examples that use ranges.

[rarkins]

Two new issues created for node pinning and ranges respectively.

Versions of npm and yarn in engines can be done like regular npmjs packages because both npm and yarn are still available via npmjs. This issue will be closed when those are supported.

[ghost]

@rarkins are still tracking support for major version upgrades in this issue?

[rarkins]

@destroyerofbuilds can you give an example of before and after values you mean?

I was tempted to close this issue after splitting everything remaining into subissues, but there are quite a few participants who won't get notifications if so.

[ghost]

can you give an example of before and after values you mean?

Yeah. I hadn't gotten to it earlier, though it has been on my todo list.

Taking over the supportPolicy option, I would like to set the following option in our group's shared configuration:

{
  "supportPolicy": ["lts_latest"]
}

Now, let's assume my projects are pinned to the latest Node 8 release, 8.9.4.

My projects are all getting deployed with Node 8.9.4.

Then, in two months, Node 10 get's released.

My projects aren't upgraded because 10 isn't LTS.

Then, in October, when Node 10 releases its first LTS version, say 10.3.0, I would like to see an upgrade pull request from Renovate upgrading the value of the node engines property from 8.9.4 to 10.3.0.

Once accepted, I would not expect to see any upgrades for Node 8 releases because those would be older than _latest LTS_.

[rarkins]

@destroyerofbuilds that's a good use case.

Once accepted, I would not expect to see any upgrades for Node 8 releases because those would be older than latest LTS.

Consider your case where you are on 8.9.4 and 10.3.0 reaches LTS status and you get a Renovate PR. Before accepting that PR, you'd still want a PR for 8.9.5 if it arrives, right?

[ghost]

I made the unstated assumption that separatemajorreleases and separatepatchreleases worked for node engine upgrades.

[rarkins]

This is now completed. There is still some improvement to be done to sync node versions with versionStrategy etc but hope to have that done by the time node@10 is LTS.