Nvm: Use package.json engines version?

[kievechua]

Run nvm use will read local package.json's engines.node and change version accordingly.
Or anyway that auto switch the version.

[ljharb]

Hmm - engines.node is only meant to be advisory, and no part of the node ecosystem uses it for any other purpose. For example, if a package specifies "0.8" that doesn't mean it won't work on iojs or 0.12 - it might be problematic to default the user to 0.8 just because the author hadn't updated their package.json.

I could definitely add this as an alternative, when .npmrc wasn't available - I'm just not convinced it would be a good thing. Also, the engines field can specify semver ranges - so I'd need to use semver to resolve that between the available node versions - which is a dependency that requires node in software that doesn't have node as a dependency.

[assaf]

NPM treats engines.node as advisory, but that's not the only way to use package.json. It's actually very convenient when everyone on the team, the CI server, and production servers all pick the same version, based on package.json. And a huge time saved when shuffling between multiple projects (consulting, microservices, migration, etc).

So it would be of great help if nvm could pick up on the current project's specific choice of iojs/Node.

[ljharb]

@assaf it already can! Just put a .nvmrc file in your project repo, and do a bare nvm install or nvm use in the project directory.

[assaf]

How do you tell .nvmrc to pick the version specified in package.json?

[ljharb]

That's not what I meant - I meant you just manually put a version in that file and commit that to the repo.

[assaf]

That's not what I was asking for. Having multiple files that specify which version of iojs/Node is in use, just means more things breaking because "works on my machine". Common sense is to keep the project DRY and have a single authoritative specification of the version in use.

[ljharb]

Right, I agree with that. The problem is that node.engines in package.json specifies the minimum version a module works with - ie, most of mine say 0.4, but I wouldn't want nvm to use 0.4 by default. By contrast, .nvmrc specifies the version I want to use it with, which is the maximum version it works with - iojs or stable for most of mine.

Thus, since they're for two separate purposes, I wouldn't want to get more DRY than two separate configurations. "Too DRY" can be worse than "not DRY enough".

[assaf]

If you don't like this feature don't use it. I'd like to have such a feature that I will then choose to use for my projects.

[ljharb]

"if you don't like it then don't use it" is not really a good justification for adding any feature to any project. It's nice that you want it, and thanks for offering your thoughts, but:

• I don't think the feature makes sense semantically - in other words, that the package.json "minimum supported version range" field makes sense as a "run it under this one single version of node/io.js"
• Parsing JSON from the shell is prohibitively complex - we can't use any npm modules and would have to use a technique that worked in bash, dash, ksh, and zsh. Can you suggest one?
• The semver range issue I mentioned above.

In other words, even if I thought this was a good feature on its face, there's just no way it's practical, so it simply can't happen.

I'm happy to review a PR in the future if someone can come up with a simple way to achieve it, but I remain opposed on the semantic appropriateness of this feature and would need to be convinced.

[assaf]

I'm most familiar with https://github.com/stedolan/jq which Heroku uses to parse package.json as part their deploy setup. Semver can also solved: https://github.com/heroku/heroku-buildpack-nodejs/blob/master/lib/build.sh#L133

PR would be pointless, we can't even agree that my use case for nvm is even worth keeping this issue open for.

[ljharb]

Yes but that would be an additional dependency to install, which currently nvm requires installing none (simply curl or wget to install node/iojs versions).

If you wrote a small commandline script that did what you wanted and supplied the version to nvm use, that would at least convince me that it's possible in a practical sense.

[assaf]

https://gist.github.com/assaf/ee377a186371e2e269a7

[ljharb]

Thanks, that's great - I'm not stoked on the reliance on a heroku app for semver resolution, and the usual advice in #iojs on freenode is to put the iojs version in engines.node, so I'd expect that to be supported.

nvm_json_extract looks solid tho, even though it's a bunch of support functions.

Currently, we have nvm-exec - what would you think about adding a new executable file to the repo that, when run, prints out the contents of engines.node or engines.iojs (and perhaps picks intelligently by default, and can be overridden to only look at a provided field)? Then at least, you could do nvm use $(nvm package-json-version) or something, and that'd be a step in the direction you want, without running afoul of my semantic concerns.

[assaf]

If you set engines.node to 1.6.2 it will call nvm use v1.6.2. If you set engines.iojs to 1.6.2 it will call nvm use iojs-v1.6.2. If you set engines.node to 0.12.0 and engines.iojs to 1.6.2, it will call nvm use iojs-v1.6.2. It follows how people use engines today to set target versions.

The semver version is used for resolving ranges. Heroku maintains it as part of their build infrastructure, a front-end to nodejs.org/iojs.org, so it's already used by many of the people who will find this feature useful, to develop and deploy against the same version.

However, if you need repeatable builds, you should set the specific version in package.json and only that version will be used. No substitutions. And no dependencies on 3rd parties.

This works. It allows you to use package.json to specify which version of io.js/Node you want your code to run against. And it's based on a common deployment pattern (Heroku but also other PaaS influenced by them).

It does not deprecate or change existing functionality. It should still be possible use .nvmrc, but this could co-exist for developer who prefer to set the engine version in package.json and have it picked up on local, CI, and production.

[ljharb]

Awesome, that behavior makes sense to me, thanks for clarifying.

Currently, if I don't have a .nvmrc and run nvm use, I get an error and help output - adding an additional fallback would be a breaking change (which isn't a problem, just worth noting).

[assaf]

I don't think it's a good idea to break current behavior. Could be a new command, or use an alias, maybe nvm use package?

[ljharb]

Interesting idea! As an alias, it'd have to be overridable, just like node/stable/unstable/iojs are, but maybe that's OK. That would certainly make it work with every nvm command seamlessly.

[assaf]

Alternatively, it could be a path, so nvm use <version | file> first looks up an alias with that name, if that fails, it looks up a file with that name. That way, it could be a shell script somewhere inside the project, pointing to the package.json in a parent directory.

And if you're in a directory that contains a package.json, nvm ls could also list that version, so you would see:

package.json -> iojs-v1.6 (-> iojs->1.6.2)

[focusaurus]

+1 to @assaf's request here. I was in the middle of coding stuff into my shell files to do exactly this when I discovered this issue. I'd also like to note that from my perspective there are big, important differences between an application and a reusable module, and many things in the npm ecosystem fail to recognize this, causing me much frustration. This is an application-driven feature request where your code is the first thing the node binary loads, and I think engines.node and engines.iojs are good ways to express an application's chosen runtime version.

[jamsyoung]

I ran across this issue when I had the same thought that it would be convenient that nvm use would just pickup whatever I have in my package.json. After reading the comments here I agree with @ljharb and can see that this will likely cause more problems than it is worth since most people probably do not use the package.json engines data like I do with CI to indicate what version of node to install on the build agent.

I would like share that you can still use the info from package.json with nvm by utilizing jq as follows:

$ nvm use $(jq -r .engines.node package.json)

And if you don't have the version installed yet, you can use the same concept with nvm install

$ nvm install $(jq -r .engines.node package.json)

Yeah, thats a lot of typing, but you could alias it if you really wanted to.

[focusaurus]

A version of grabbing engines.node without the external dependency on jq is: node -p -e 'require("./package").engines.node'

[ljharb]

Although that won't work unless you already have node installed, which isn't something to rely on when using a node version manager :-)

[bettiolo]

Is anyone actively working on a command line nvm < use | install > package.json?

[hurrymaplelad]

FYI, nodenv has a passable plugin api copied from rbenv. It was pretty straightforward to build a plugin to parse package.json engines after @assaf did all the hard work :)

We're looking at using sh-semver to evaluate semver ranges without a network call or a node dependency.

[patcon]

Just wanted to chime in to say that I agree with @assaf's original sentiment that locking to specific node version should be encouraged -- lack of version constraint is a good thing imho, should folks choose to use engines.node

[acjay]

:+1: To the original suggestion. It's super helpful in having a smooth process for getting people up and running on a project locally, especially from zero. I'm imagining a workflow that would be as simple as:

• clone project from Github
• install NVM
• nvm install package.json or something, which would install the correct version of Node and NPM for the project and npm install all the modules
• npm start

Even better, if there were a way to nvm install-repo [email protected]:..., we could get rid of step 1.

[ljharb]

@acjay that's already doable if you put .nvmrc in the github repo. then you can just run nvm install and/or nvm use and it will work.

[patcon]

:+1: @ljharb I somehow missed that. Still feel package.json or npm-shrinkwrap.json (latter would need a core issue) would be suitable, but that's otherwise very helpful. Thanks!

[acjay]

@ljharb Ah nice! So then we're just a short hop away from my dream scenario :)

But thanks for the info, I can definitely live with a .nvmrc.

[ljharb]

The difficulty remains in using POSIX to read JSON, with no node available ¯\_(ツ)_/¯

[acjay]

Good point!

[geophree]

@ljharb haven't used it, but there's this: "Pure-POSIX-shell JSON parser." https://github.com/rcrowley/json.sh (BSD-licensed)

[tomdavidson]

This workaround assumes node, but would help keep .nvmrc in sync with package.json:
"scripts": { "preinstall": "echo $(node -p -e 'require(\"./package\").engines.node') > .nvmrc && nvm install && nvm use",

But why would nvm not be found in the sub-shell (works fine outside of npm run)?
> echo $(node -p -e 'require("./package").engines.node') > .nvmrc && nvm install && nvm use sh: 1: nvm: not found

[brettneese]

I still think this is a good idea. I understand this is the point of an .nvmrc but our repos are already cluttered with dotfiles, and it seems silly to add one more just for this when there's a place for that in the package.json (which, to me, ought to be the singular point of truth for all package configuration)

[ljharb]

@brettneese engines points to a semver range, and is and has always been purely advisory. I understand that you want the engines field to be proscriptive, but nvm shouldn't be the first tool to respect it by default (not counting warnings).

[brettneese]

There's also engineStrict, which is prescriptive

[ljharb]

Which npm 3 and later have removed, because it turns out to be very bad for the ecosystem. engineStrict is dead.

[brettneese]

Sure, ok. But straight from the docs:

You can specify the version of node that your stuff works on

Wouldn't it make sense that "the version of node that your stuff works on" is the version that NVM uses?

https://docs.npmjs.com/files/package.json#engines

[ljharb]

The docs are inaccurate there - you can specify the versions of node that your stuff works on.

nvm doesn't work with semver ranges, it only works with a full or partial single version string - which can't be inferred from the "engines" field without a semver parser.

[brettneese]

That, in and of itself, is not terribly difficult due to the very nature of semver. Cloudflare did it in bash here: https://github.com/cloudflare/semver_bash/blob/master/semver.sh (Not that I particularly trust anything Cloudflare does anymore ;-)) We hard-code our node versions so that's a non-issue for us but I understand that would need to be in place.

I might even be willing to hack this together at some point and put in a PR to stop bikeshedding (I don't expect project owners to take care of all user requests), but given the response so far I'm not sure it'd be accepted anyway, and I'm still not sure I understand why. It does seem like that's exactly what the field is for and some deployment engines even use it (I think Heroku does, for instance, but it's been ages since I've used Heroku.)

Right now, if we were to properly follow convention, updating our node version for our project would require updating _4_ files, at least (.nvmrc, Dockerfile, package.json, readme.) I found and responded to this issue because our package.json was out-of-date and I was having some issues, because, honestly, I thought package.json was our primary point of truth.

Yes, the docs also say it's advisory, but if our tools don't make our jobs easier and our workflow less convoluted, then what's the point in having them at all?

[ljharb]

I'm always open to reviewing a PR! https://github.com/creationix/nvm/issues/651#issuecomment-85827779 is an idea to make it explicit.

I agree that having to update 4 files seems excessive - i'm not sure why the readme would mention a node version tho, nor why the Dockerfile wouldn't read it out of package.json or nvmrc?

[edwmurph]

I implemented it in bash here:
https://github.com/creationix/nvm/pull/1535

[cyrilchapon]

@brettneese engines points to a semver range, and is and has always been purely advisory. I understand that you want the engines field to be proscriptive, but nvm shouldn't be the first tool to respect it by default (not counting warnings).

Actualy, this is not. Heroku pulls node and npm version from here by default and design (before and among anywhere else)

[ljharb]

@cyrilchapon since it's a range, does it pick the latest in the range?

[cyrilchapon]

I don't have a clue, but I think one could find the strategy (and probably replicate it) inside heroku repos ?

Edit : probably inside https://github.com/heroku/heroku-buildpack-nodejs
Edit2: in here https://github.com/heroku/heroku-buildpack-nodejs/blob/4cf68e4e63698cdb56ee18737533482d99d25a74/lib/binaries.sh

[ljharb]

Ah, their strategy requires a running server to resolve the semver range; that won’t work for nvm. Good to have the data point, tho!

[patcon]

Does it? Isn't it just polling an endpoint? Or am I misunderstanding?

JSON:
https://nodebin.herokai.com/v1/node/linux-x64/latest?range=8.x
https://nodebin.herokai.com/v1/node/linux-x64/latest?range=<8.2
https://nodebin.herokai.com/v1/node/linux-x64/latest?range=>8.x
https://nodebin.herokai.com/v1/node/linux-x64/latest?range=^8.2
https://nodebin.herokai.com/v1/node/linux-x64/latest?range=~8.2

Or txt: https://nodebin.herokai.com/v1/node/linux-x64/latest.txt?range=8.x

[ljharb]

That, itself, is a list of endpoints.

[patcon]

I am confused, but that's ok -- this is a long thread. I'm assuming your response is short-hand for saying that a half-way-there UX improvement (that piggybacks on Heroku's semver-resolution strategy and infra) is a show-stopper since it only addresses non-offline users :)

If so, then I feel differently, but genuinely respect your decision!

[ljharb]

@patcon yes, definitely - it's more than just offline users, tho (since nvm ls-remote is also a factor) but also that I don't want to add yet another remote dependency and network call. Proxies already pose problems for nodejs.org/iojs.org; it would be a burden to these users to be forced to add, or create, yet another mirror for a "semver API".

[cyrilchapon]

Wouldn't that be OK just to bind-to-http-resolution the nvm install (and nvm ls-remote), using package.json[engines] and somehow keep the nvm use local (without HTTP call) with a clever algorithm processing semver-specified release in package.json against installed releases.

I mean, after all nvm install will definately make some HTTP roundtrips, and "processing a semver against a list of installed versions" wouldn't be so hard, would it ?

[ljharb]

@cyrilchapon See https://github.com/nvm-sh/nvm/issues/651#issuecomment-455380459

[mflodin]

Would it be possible to do something like eslint-plugin-react does for React version - i.e. specify in the config file that you want to use the version specified in package.json?

// .eslintrc
{
  "settings": {
    "react": {
      "version": "detect"
    }
  }
}

Perhaps something like this:

// .nvmrc
engines.node // or whatever the property is in package.json

That could also avoid the problem of semver ranges by simply not allowing it if you wish to use this feature, as this would make the feature opt in.

[mrchief]

That could also avoid the problem of semver ranges

Or you could specify the range as well, kinda like browserlist config.

[jasonrberk]

just to compare....N does this already....according to their docs

the idea of having to specify my node version in two places is a non-starter. No way that _won't_ fall out of sync.

[pmrotule]

It would be really nice to add this feature:

• First check if .nvmrc is present and use that version if the file exists (current behavior)
• If not, check for engines.node config in package.json

Supporting strict version (i.e. "12.18.3") should be easy to implement and already a good start.

[koresar]

Alternatively, it's not necessary to use "engines" property. We can have our own "nvmrc" property in package.json.

The package.json can have a special field called "nvmrc" which would contain the NVM's .nvmrc file contents. It's quite easy to parse package.json with grep.

Here is how you retrieve package version (cross platform version of grep): grep -o '"version": "[^"]*' package.json | grep -o '[^"]*$'

So, NVM could check if .nvmrc file presents OR "nvmrc": "lts" value is present in package.json.

Here is bash POC:

if test -f ".nvmrc"; then
  NODE_VERSION = `cat .nvmrc || ""`
elif test -f "package.json"; then
  NODE_VERSION = `grep -o '"nvmrc": "[^"]*' package.json | grep -o '[^"]*$' || echo ""`
else
  NODE_VERSION = ""
fi

[ljharb]

@koresar sure, an nvmrc property in package.json wouldn't be a range, it'd just be the same as .nvmrc, except then nvm would still have to have a JSON parser. Why is that better than using a .nvmrc file?

[koresar]

1. Nope. No need for JSON parser. See my POC above.
2. Why people want to put .nvmrc to package.json? There are answers above in this thread. My personal use case - our deployment scripts do not copy dot-files for security reasons.

[ljharb]

@koresar many legit things live in dotfiles; that seems like a huge flaw in your deployment scripts. The majority of the answers upthread are about having a single source of truth - having two fields in the same file doesn't achieve that.

re your POC, that won't work unless there's a single space after "nvmrc":, which isn't guaranteed, and it doesn't handle a bunch of other whitespace characters that are perfectly valid in JSON. Anything that touches JSON without being a proper JSON parser is automatically broken.

[koresar]

@ljharb good job spotting the "space problem" in my POC. You are good with RegEx. 👍
So you probably know that this
grep -o '"version":\s*"[^"]*' package.json | grep -o '[^"]*$' || echo ""
would work with any number of space or tab characters. With some more lines of code we could make this work for new-lines as well.

I find your reply deliberately misleading and unproductive. Somehow you are looking for the proofs to tell other people are wrong, rather than collaborate effectively.

So, again, we can safely parse out single string out of a JSON file using RegEx.
Moreover, I believe that there is a 99.9999% chance that this grep would work as is. Which is high enough for production usage.

We already have two sources of truth - package.json and .nvmrc files. How come joining two sources into single source of truth doesn't achieve the requested feature?

And lastly, our deployment scripts "huge flaw" is kinda more difficult than you can imagine. There is more to that. Please, avoid basing your conclusions on my rather not informative sentence. I can' disclose more information at this point. Sorry.

Have a good day.

[ljharb]

JSON isn't regular, so no, regular expressions simply aren't safe. I appreciate that it would work 99% of the time, but if it's not 100%, it's not safe enough for nvm.

[koresar]

Mate, not 99%. I said 99.9999%.

Please, avoid faking other people words. It's unproductive.

Thanks.

[koresar]

Good news everyone! There is a JSON parser written in pure shell script.

https://github.com/dominictarr/JSON.sh

Here is how I can extract any value from a JSON file:

cat package.json | JSON.sh -b | grep '\[\"version\"\]'

This means that NVM can 100% safely keep its version in the ["nvmrc"] (root) value of the package.json.

Who else still wants to keep the nvmrc value inside the package.json? I do. :)

[ljharb]

@koresar ok, 99.99999999999% is still not sufficient. It's 100% or nothing. I wasn't "faking" any words, I was approximating/paraphrasing, which is a normal thing humans do in conversation. Please don't be overly pedantic, it's unproductive. Thanks.

[ljharb]

Unfortunately, at a quick glance that tool isn't compatible with ksh due to its use of local declarations on the same line as a variable initialization.

I agree that using a full JSON parser is the only safe approach, and that one's pretty close to what we'd need - but it's still not clear that the complexity and bytes weight is a good tradeoff for being able to have your two sources of truth in the same file.