Yarn: Error: Couldn't find the binary node install.js in 1.0.1

[georeith]

Our CI pulls in the latest version of yarn and installs it on each build. It's just started failing in a project which installs eslint and runs the following command:

yarn run lint -- --max-warnings=0

The script definition:

{  "lint": "eslint ./" }

And gives the error output:

$ eslint ./ "--max-warnings=0"
error Couldn't find the binary eslint ./ "--max-warnings=0"

The version it is failing with is 1.0.1. I am unaware of whether 1.0.0 also fails as it never attempted to install and run it.

EDIT: Clearing all caches on the CI server and running again results in the following errors:

Couldn't find the binary node-gyp rebuild
Couldn't find the binary node install.js

I believe $PATH is not getting set up correctly via the install script now. This is running on Ubuntu 12.04 and installed via:

curl -o- -L https://yarnpkg.com/install.sh | bash

That or if the error message is to believed it seems to be treating the entire command as a binary and failling to find it?

[Vanuan]

Similar issue in install/postinstall scripts:

 [4/4] Building fresh packages...
 error /src/node_modules/husky: Couldn't find the binary node ./bin/install.js

After being run several times it's a different error.
I suspect the issue is not $PATH. Too bad yarn doesn't print the error message from the command. Are there any logs?

[joeporpeglia]

@georeith what node version are you running? I'm experiencing the same issue with yarn 1.0.1 and node 4.2.4, but can't reproduce using node 8.2.1. Still not sure what could cause this though...

[cwarden]

I'm seeing the error with node 4.5.0, but not 6.11.3.

[BYK]

This is because we have started to use shell: true in #4031 which was supposed to be available in Node 4 but apparently it was introduced sometime around 4.4 or later. The latest 4.x which is 4.8 should pass since that's apparently what our CI builds use.

Is it possible for you to upgrade your Node to latest 4.x version?

[Vanuan]

I can reproduce this with 4.7

[Vanuan]

Here's another error example:

error /src/node_modules/radium: Couldn't find the binary cd lib || npm run lib

[cameronwp]

Upgrading node from 4.4.5 to 4.8.4 fixed it for me.

[Vanuan]

Yeah, I can confirm that upgrade from 4.7 to 4.8 solved the issue.

[georeith]

@joeporpeglia @BYK Our CI is using node 5.4.0

[MatissJanis]

My CI is using node 5.5.0 and it's failing with this error.

[Vanuan]

Looks like the minimum version requirements are 6.0, 5.7 and 4.8.

https://nodejs.org/en/blog/release/v4.8.0/

child_process: add shell option to spawn() (cjihrig) #4598

https://nodejs.org/en/blog/release/v5.7.0/

child_process: spawn() and spawnSync() now support a 'shell' option to allow for optional execution of the given command inside a shell. If set to true, cmd.exe will be used on Windows and /bin/sh elsewhere. A path to a custom shell can also be passed to override these defaults. On Windows, this option allows .bat. and .cmd files to be executed with spawn() and spawnSync(). (Colin Ihrig) #4598

[BYK]

@Vanuan thanks a lot for digging up the version numbers!

@georeith @MatissJanis is this information useful for you? If so can we close the ticket? If not any proposal for a potential fix?

[ipsquare]

+1 for request to fix

I am using same Node version, dependencies, code etc. and can reproduce this issue with [email protected].

Reverting back to [email protected] fixes this magically so I think is a bug and fix will help Yarn to be used with wide range of Nodejs versions, like pre-upgrade.

[Vanuan]

@ipsquare

I am using same Node version

There were at least 3 versions of Node mentioned. It was specified which of them worked and which you should upgrade to.

Reverting back to [email protected] fixes this magically so I think is a bug

This it not a bug. This is a deliberate breakage of compatibility with older versions of Node.

P.S. The oldest version of Node supported (4.0.0) was originally released 2 years ago. An upgrade to it was released 7 months ago (4.8.0). It's currently in maintenance mode. End of life is planned in 7 months.

P.P.S. Maintenance doesn't mean that something that didn't work in previous version will work in a new one. So it doesn't mean you shouldn't upgrade. It only means that there will be no breaking changes so it's safe to upgrade.

P.P.P.S. In October will be a start of Node 8.x LTS. I think it's time to upgrade. See this schedule to plan your upgrades: https://github.com/nodejs/LTS

[dmenounos]

Hi, please consider that using yarn with nodejs on a recent ubuntu linux does not work anymore due to this issue. Currently the nodejs version in the ubuntu repositories is 4.7.2.

[Vanuan]

AFAIK, yarn is supposed to be used only by developers, not by end users. And developers don't use node versions coming from OS repositories. They usually use nvm (Node Version Manager) or Docker. It's kind of silly to rely on your host OS to match the version of node you're using in your project.

[dmenounos]

Hi Vanuan. I think that is half true. The official and recommended way to install yarn is through the package manager of the distribution. Of course that should also be the case for nodejs as it is a dependency of yarn. See the following:

https://yarnpkg.com/en/docs/install#alternatives-tab
"On common Linux distributions such as Debian, Ubuntu and CentOS, it is recommended to install Yarn via our packages instead."

So, given that recommendation, it is not cool when "stable" software stops working after a regular system update.

[BYK]

@Vanuan that's not a very accurate assumption. We expect Yarn to be used in CI environments too.

@dmenounos I do understand the pain here. That said Yarn 1.0 was a major release with some breaking changes in it. You can consider this as one of them. We are committed to providing stability and backward-compatibility as long as possible. That said trying to support very old Node versions that are about to go out of LTS puts unnecessary maintenance burden on the project.

Is it not possible for you to at least upgrade to latest Node 4 version? As far as I know, even Node folks suggest using a custom repo for installing node: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions and they don't even list Node 4 there anymore.

[Vanuan]

@dmenounos

"On common Linux distributions such as Debian, Ubuntu and CentOS, it is recommended to install Yarn via our packages instead."

Yeah, but what is "our packages" requirements? When you get to docs on how to install Yarn "via our packages" you'll see this:

On Ubuntu 16.04 or below and Debian Stable, you will also need to configure the NodeSource repository to get a new enough version of Node.js.

So you won't use Node from official package of distribution. Probably my wording wasn't clear. I meant that anybody who does development doesn't use programming language environment that comes with mainstream OS. If you do JS development you're gonna live on the edge.

@BYK

We expect Yarn to be used in CI environments too.

Yeah, but CI is also used by developers, not by end users. And CI is usually set up by developers, because you can't reasonably expect operations team to set up package version for each project. Unless you're in a really big company. Maybe you get another experience at Facebook.

I think most CI systems are self-served. You can either:
1) use the same version of node for all project. Not an option, because in the transition period you'll need at least 2 versions
2) use multiple virtual machines with different node versions installed (via puppet, chef, ansible, etc). That way yeah, you'll get versions you need, but you'll need infrastructure to look in project's repository which version is required and then redirect the build to another virtual machine
3) use multiple node versions installed for different projects. Probably the most common option. Not sure how it's possible to install multiple versions of a package with a same name. Probably you'll get custom build packages with names node4, node6, node8. Still it's a huge pain to set up proper PATH and LD_LIBRARY_PATH.
4) use containers. Again customly build. If you go to Node's Docker Hub you'll see that each version of node is downloaded from a precompiled location, not installed via package manager.

As you can see, only in option 2 you'll probably benefit from OS package manager, but even then you have no control over a specific version and whenever OS maintainer decide to remove the version you use from repository, you'll see failures in your CI. So you'd have to customize it and use NodeSource and/or package mirrors.

Sorry if it's an off-topic.

[BYK]

I'll go ahead and close this since I think we have a mutual understanding and I don't have any intentions to revert that shell patch only to support very old versions of Node.

Feel free to reopen if you disagree and would like to discuss more.