Docker-node: yarn not executable

[hjanuschka]

using :9 image.

recent changes made yarn not executable.

/usr/local/bin/yarn: Permission denied

chmod a+rwx  /usr/local/lib/node_modules/yarn/bin/yarn*
chmod a+rwx  /usr/local/bin/yarn*

as a workaround fixes it

[SimenB]

I cannot reproduce this:

$ docker run --rm node:9 yarn --version
Unable to find image 'node:9' locally
9: Pulling from library/node
f2b6b4884fc8: Pull complete
4fb899b4df21: Pull complete
74eaa8be7221: Pull complete
2d6e98fe4040: Pull complete
452c06dec5fa: Pull complete
7b3c215894de: Pull complete
c2139be2d901: Pull complete
7c9b0e13f02b: Pull complete
Digest: sha256:59aff2d9898e2b3d35d7656bf57bf61c2dcaedac58b8c3934b8963904d27c437
Status: Downloaded newer image for node:9
1.5.1

Are you sure you're on the latest?

[hjanuschka]

very strange, i could have sworn that i reproduced it on ci.

docker run --rm node:9 sh -c 'npm i -g yarn && yarn -v'

now, back home, i am unable to reproduce it, was there a faulty built in between yesterday and today?

sorry for the noise.

[miltonbecker]

@hjanuschka @SimenB
Hey, I just ran into this problem too and I found out why.
On the previous node images, the pre-installed yarn was so old (version 0.x) we had to do a npm i -g yarn to get the latest version.

Now we already get the latest version of yarn pre-installed, so there's no need for it anymore.

However, if we still do npm i -g yarn, we get the permission denied error.
So, I guess this is still an issue, even though it's a minor one.

[hjanuschka]

@miltonbecker awesome explanation thx!

but running

docker run --rm node:9 sh -c 'npm i -g yarn && yarn -v'

today does not break anymore, can you reproduce it?

in my large projects, ci / dockerfile - i ended up doing the chmod everwhere.
and yes, the image should not fail if one decides to npm i -g yarn

[miltonbecker]

@hjanuschka yep, just reproduced it with node:latest.
On node:9.7 I don't get the error.

Alternatively, removing npm i -g yarn fixes it for node:latest.

Edit: btw, just tested the chmod +x /usr/local/bin/yarn and that works too for node:latest, thanks for that :)

[hjanuschka]

thats strange:

docker run --rm node:latest sh -c 'npm i -g yarn && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.991s
1.5.1

seems to work
@SimenB any ideas?

[miltonbecker]

@hjanuschka try removing your local image to make sure you get the latest one: docker rmi node:latest

Then run docker run --rm node:latest sh -c 'npm i -g yarn && node -v && yarn -v'

Edit: this is what I get:

~ docker run --rm node:latest sh -c 'npm i -g yarn && node -v && yarn -v'
Unable to find image 'node:latest' locally
latest: Pulling from library/node
f2b6b4884fc8: Pull complete
4fb899b4df21: Pull complete
74eaa8be7221: Pull complete
2d6e98fe4040: Pull complete
452c06dec5fa: Pull complete
7b3c215894de: Pull complete
c2139be2d901: Pull complete
7c9b0e13f02b: Pull complete
Digest: sha256:59aff2d9898e2b3d35d7656bf57bf61c2dcaedac58b8c3934b8963904d27c437
Status: Downloaded newer image for node:latest
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.579s
v9.8.0
sh: 1: yarn: Permission denied

[chorrell]

I can't reliably reproduce this. Occasionally if fails, but it also succeeds:

$ docker run --rm node:latest sh -c 'npm i -g yarn && node -v && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.713s
v9.8.0
1.5.1

Doing this (omitting node -v) seems to fail 50% of the time:

$ docker run --rm node:latest sh -c 'npm i -g yarn && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.655s
1.5.1

I wonder if it's an issue with timing?

[chorrell]

Following up on my previous comment, if I add a sleep after the npm i it always succeeds:

 $ docker run --rm node:latest sh -c 'npm i -g yarn;sleep 5; yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.684s
1.5.1

[miltonbecker]

@chorrell you're right! I couldn't always reproduce it here either.

But, if you try it on node:9.7, you can see it always works: docker run --rm node:9.7 sh -c 'npm i -g yarn && yarn -v'

I've checked the npm version on both images and they're the same (5.6.0), so that's not it.
There's something funky on the node:9.8 image.

[miltonbecker]

@hjanuschka @SimenB @chorrell : should this issue be re-opened? What do you think?

[chorrell]

I've been running the following command and occasionally it does fail:

$ docker run -it --rm node:9.9.0 sh -c 'npm i -g yarn && node -v && ls -AlFh /usr/local/lib/node_modules/yarn/bin/ && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.692s
v9.9.0
total 20K
-rwxr-xr-x 1 root root 920 Feb 26 19:00 yarn*
-rwxr-xr-x 1 root root  34 Feb 26 19:00 yarn.cmd*
-rw-r--r-- 1 root root 907 Mar 22 00:58 yarn.js
-rwxr-xr-x 1 root root  42 Feb 26 19:00 yarnpkg*
-rwxr-xr-x 1 root root  30 Feb 26 19:00 yarnpkg.cmd*
sh: 1: yarn: Permission denied

Note the permissions on /usr/local/lib/node_modules/yarn/bin/yarn.js compared to a successful run:

$ docker run -it --rm node:9.9.0 sh -c 'npm i -g yarn && node -v && ls -AlFh /usr/local/lib/node_modules/yarn/bin/ && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.76s
v9.9.0
total 20K
-rwxr-xr-x 1 root root 920 Feb 26 19:00 yarn*
-rwxr-xr-x 1 root root  34 Feb 26 19:00 yarn.cmd*
-rwxr-xr-x 1 root root 907 Mar 22 01:00 yarn.js*
-rwxr-xr-x 1 root root  42 Feb 26 19:00 yarnpkg*
-rwxr-xr-x 1 root root  30 Feb 26 19:00 yarnpkg.cmd*
1.5.1

I'm not sure why but occasionally npm i -g yarn on the node:9.9.0 produces a yarn.js that's not executable. Is this maybe some kind of exotic npm bug that happens under docker?

I occasionally hit this with node:8 too:

$ docker run -it --rm node:8 sh -c 'npm i -g yarn && node -v && ls -AlFh /usr/local/lib/node_modules/yarn/bin/ && yarn -v'
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.628s
v8.10.0
total 20K
-rwxr-xr-x 1 root root 920 Feb 26 19:00 yarn*
-rwxr-xr-x 1 root root  34 Feb 26 19:00 yarn.cmd*
-rw-r--r-- 1 root root 907 Mar 22 01:05 yarn.js
-rwxr-xr-x 1 root root  42 Feb 26 19:00 yarnpkg*
-rwxr-xr-x 1 root root  30 Feb 26 19:00 yarnpkg.cmd*
sh: 1: yarn: Permission denied

I can't seem to reproduce this with node:6...

I think the version of npm is relevant:

$ docker run -it --rm node:6 sh -c 'npm --version'
3.10.10
$ docker run -it --rm node:8 sh -c 'npm --version'
5.6.0
$ docker run -it --rm node:9 sh -c 'npm --version'
5.6.0

[chorrell]

For now, a simple workaround is to make sure yarn.js is executable after the install:

npm i -g yarn && chmod +x /usr/local/lib/node_modules/yarn/bin/yarn.js

[hjanuschka]

hmmm anyway it makes the image unreliable.

why is it installed this way:

RUN set -ex \
  && for key in \
    6A010C5166006599AA17F08146C2130DFD2497F5 \
  ; do \
    gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys "$key" || \
    gpg --keyserver hkp://ipv4.pool.sks-keyservers.net --recv-keys "$key" || \
    gpg --keyserver hkp://pgp.mit.edu:80 --recv-keys "$key" ; \
  done \
  && curl -fSLO --compressed "https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v$YARN_VERSION.tar.gz" \
  && curl -fSLO --compressed "https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v$YARN_VERSION.tar.gz.asc" \
  && gpg --batch --verify yarn-v$YARN_VERSION.tar.gz.asc yarn-v$YARN_VERSION.tar.gz \
  && mkdir -p /opt \
  && tar -xzf yarn-v$YARN_VERSION.tar.gz -C /opt/ \
  && ln -s /opt/yarn-v$YARN_VERSION/bin/yarn /usr/local/bin/yarn \
  && ln -s /opt/yarn-v$YARN_VERSION/bin/yarnpkg /usr/local/bin/yarnpkg \
  && rm yarn-v$YARN_VERSION.tar.gz.asc yarn-v$YARN_VERSION.tar.gz

instead of just using npm? - i guess its because the symlink already exists - the npm installer places the yarn.js in bin, but does ignore the step where it does chmod. and the yarn.js is missing the chmod coz it would get it via the symlink. if the symlink wouldn't exist

if it is installed via npm i - bet the detection that yarn is already installed globally would work.

[SimenB]

Installing yarn via npm is not recommended

[hjanuschka]

@SimenB https://yarnpkg.com/en/docs/install#alternatives-tab perfectly explains that, didn't know that.
however - i think the image should not "destroy" yarn - if users are using npm to install it :/

[miltonbecker]

Installing yarn via npm is not recommended because they think it's not secure enough, not because it shouldn't work.

Installation of Yarn via npm is generally not recommended. When installing Yarn with Node-based package managers, the package is not signed, and the only integrity check performed is a basic SHA1 hash, which is a security risk when installing system-wide apps.

Also, installing it via npm makes sure we always get the latest version of yarn, instead of depending on the image's version or the distro's repo.

Anyway, what we have so far:

• node:9.7: works
• node:9.8: unstable
• node:9.9.0: unstable
• node:6: works
• node:8: unstable

I understand this is no simple bug and there are simple workarounds available already, so I guess we might have to be happy with those even though yarn is getting more and more popular.

[Maxwell2022]

This is working for me as a work around using Docker and node:8 image:

FROM node:8

# Remove the version of yarn that is coming with node:8 & Install latest yarn
RUN rm -f /usr/local/bin/yarn && \
  curl -o- -L https://yarnpkg.com/install.sh | bash && \
  chmod +x ~/.yarn/bin/yarn && \
  ln -s ~/.yarn/bin/yarn /usr/local/bin/yarn

[AndrewRayCode]

I'm hitting this too across projects running in CircleCI, with tags circleci/ruby:2.5.1-node-browsers and circleci/node:8.11.1. It's intermittent. Will try the above fix for now. I'm not sure if both of these are based off the official docker-node images?

[LondonAppDev]

I was hitting this when using node:8.11-alpine and installing yarn with NPM.

I didn't realise yarn was included with the base image, so I removed npm install -g yarn and it solved the issue.

[fpietka]

I found out adding a small pause after installing yarn avoid getting the yarn: Permission denied.
Could it be that yarn installation did not have enough time to add the executable flag?

[raphaelsoul]

Same problem and also not reliably reproduce. Tried install with official install script, causes another problem.

see the logs.

Running with gitlab-runner 11.1.0 (081978aa)
  on 88221060076d d055fb60
Using Docker executor with image node:8 ...
Pulling docker image node:8 ...
Using docker image sha256:ed145ef978c437fb4e0fbc117b2df891767dc55067124fc56b85aeec97f827ff for node:8 ...
Running on runner-d055fb60-project-7-concurrent-0 via 88221060076d...
Fetching changes...
HEAD is now at 4e982d3 Update .gitlab-ci.yml
warning: redirecting to https://git.XXXX.com/XXXX/XXXX.git/
From http://git.XXXX.com/XXXX/XXXX
   4e982d3..a583dfb  master     -> origin/master
Checking out a583dfba as master...
Skipping Git submodules setup
Checking cache for default...
Successfully extracted cache
$ curl -o- -L https://yarnpkg.com/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  7041    0  7041    0     0   5933      0 --:--:--  0:00:01 --:--:--  5936
Installing Yarn!
> Downloading tarball...

[1/2]: https://yarnpkg.com/latest.tar.gz --> /tmp/yarn.tar.gz.m8gyuOLx82
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   608    0   608    0     0    257      0 --:--:--  0:00:02 --:--:--   531
100  922k  100  922k    0     0  77364      0  0:00:12  0:00:12 --:--:--  176k

[2/2]: https://yarnpkg.com/latest.tar.gz.asc --> /tmp/yarn.tar.gz.m8gyuOLx82.asc
100   832  100   832    0     0    798      0  0:00:01  0:00:01 --:--:--  812k
> Verifying integrity...
gpg: Signature made Wed May 23 21:02:50 2018 UTC using RSA key ID B6FF4DE3
gpg: Good signature from "Yarn Packaging <[email protected]>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 72EC F46A 56B4 AD39 C907  BBB7 1646 B01B 86E5 0310
     Subkey fingerprint: E219 30C4 D0A4 AA46 1858  1F7A E074 D16E B6FF 4DE3
> GPG signature looks good
> Extracting to ~/.yarn...
> Adding to $PATH...
> We've added the following to your /root/.bashrc
> If this isn't the profile of your current shell then please add the following to your correct profile:

export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"

> Successfully installed Yarn 1.7.0! Please open another terminal where the `yarn` command will now be available.
$ node -v && npm -v && yarn -v
v8.11.3
5.6.0
1.6.0
$ echo "test pipeline runs good!"
test pipeline runs good!
Creating cache default...
WARNING: node_modules/: no matching files          
Archive is up to date!                             
Created cache
Job succeeded

Notice that installed version is 1.7 but outputs yarn version 1.6.0 🤣

[mhermosi]

I am having this issue running a container on Atlassian bitbucket pipeline. We are using a maven plugin to execute node builds, with node 9.4.0 and yarn 1.8.0 and the error is:

yarn: exec: line 32: node: Permission denied

I am trying switching node to 9.7 as seen in older comments.

[raphaelsoul]

As I realized yarn is built-in in node image. I think this issue could be closed

[raphaelsoul]

@mhermosi you can use two images to build node and java code parallel

[majestica]

I have same problem with exec: line 32: node: Permission denied
yarn 1.9.4
I use docker gitlab runner with maven:3.3-jdk-8-alpine image
sombody already found a solution to fix this error?
Thanks

[AmlingPalantir]

I can reproduce this, but not with --no-bin-links. Reading the bin links code it seems not ready for two links pointing to the same place (as the yarn package does): e.g. its dos2Unix function will be run two async, each stat()ing the file, then rewriting it, then rechmoding it. The "right" interleave (stat1, rewrite1, stat2 (gets bad perms), chmod1, rewrite2, rechmod2) will result in lost permissions.

[chorrell]

Closing, since I believe this us ultimately an issue of installing yarn when it's already available. Also note that update yarn as pretty frequently now.

[Vanuan]

Here are reproduction steps:

# you need 3 to 10 runs to reproduce it once
docker rm npm_run; docker run -ti --name npm_run node:8.11-alpine sh -c "npm install -g [email protected] && yarn config set prefix /usr/local"

After reproducing the issue I inspected the container:

docker commit npm_run testimage
docker run -ti --rm testimage sh

/ # ls -al  /usr/local/lib/node_modules/yarn/bin/
total 28
drwxr-xr-x    2 root     root          4096 Mar 19 10:08 .
drwxr-xr-x    4 root     root          4096 Mar 19 10:08 ..
-rwxr-xr-x    1 root     root          1025 Dec 18 16:04 yarn
-rwxr-xr-x    1 root     root            34 Dec 18 16:04 yarn.cmd
-rw-r--r--    1 root     root          1015 Mar 19 10:08 yarn.js
-rwxr-xr-x    1 root     root            42 Dec 18 16:04 yarnpkg
-rwxr-xr-x    1 root     root            30 Dec 18 16:04 yarnpkg.cmd
/# ls -al  /usr/local/bin/
total 35488
drwxr-xr-x    1 root     root          4096 Mar 19 10:08 .
drwxr-xr-x    1 root     root          4096 Mar 19 10:08 ..
-rwxr-xr-x    1 root     root      36329336 Sep  4  2018 node
lrwxrwxrwx    1 root     root            38 Sep  4  2018 npm -> ../lib/node_modules/npm/bin/npm-cli.js
lrwxrwxrwx    1 root     root            38 Sep  4  2018 npx -> ../lib/node_modules/npm/bin/npx-cli.js
lrwxrwxrwx    1 root     root            36 Mar 19 10:08 yarn -> ../lib/node_modules/yarn/bin/yarn.js
lrwxrwxrwx    1 root     root            36 Mar 19 10:08 yarnpkg -> ../lib/node_modules/yarn/bin/yarn.js

So it doesn't look like sleep should help (And it didn't help, I tried). The issue is that /usr/local/lib/node_modules/yarn/bin/yarn.js doesn't have the correct permissions.

Here's what yarn links to before installing yarn:

/ # ls -al /usr/local/bin/yarn
lrwxrwxrwx    1 root     root            25 Sep  4  2018 /usr/local/bin/yarn -> /opt/yarn-v1.6.0/bin/yarn

Let's try removing this link before installing yarn. Doesn't help either:

docker rm npm_run; docker run -ti --name npm_run node:8.11-alpine sh -c "rm /usr/local/bin/yarn && npm install -g [email protected] && yarn config set prefix /usr/local"
npm_run
/usr/local/bin/yarn -> /usr/local/lib/node_modules/yarn/bin/yarn.js
/usr/local/bin/yarnpkg -> /usr/local/lib/node_modules/yarn/bin/yarn.js
+ [email protected]
added 1 package in 0.858s
sh: yarn: Permission denied

So the bug appears to be in npm itself.

[Vanuan]

Moreover, the yarn.js file from npm package is being modified after downloading:

/ # ls -al /usr/local/lib/node_modules/yarn/bin
total 28
drwxr-xr-x    2 root     root          4096 Mar 19 11:16 .
drwxr-xr-x    4 root     root          4096 Mar 19 11:16 ..
-rwxr-xr-x    1 root     root          1025 Dec 18 16:04 yarn
-rwxr-xr-x    1 root     root            34 Dec 18 16:04 yarn.cmd
-rw-r--r--    1 root     root          1015 Mar 19 11:16 yarn.js     <---- modified date
-rwxr-xr-x    1 root     root            42 Dec 18 16:04 yarnpkg
-rwxr-xr-x    1 root     root            30 Dec 18 16:04 yarnpkg.cmd

/ # ls -al yarn-v1.13.0/bin/
total 28
drwxr-xr-x    2 root     root          4096 Mar 19 11:17 .
drwxr-xr-x    4 root     root          4096 Mar 19 11:17 ..
-rwxr-xr-x    1 root     root          1025 Dec 18 16:04 yarn
-rwxr-xr-x    1 root     root            34 Dec 18 16:04 yarn.cmd
-rwxr-xr-x    1 root     root          1015 Dec 18 16:04 yarn.js     <---- normal date
-rwxr-xr-x    1 root     root            42 Dec 18 16:04 yarnpkg
-rwxr-xr-x    1 root     root            30 Dec 18 16:04 yarnpkg.cmd

Might it be yarn itself?