yarn install fails with `ENOENT: no such file or directory` occasionally

[NiGhTTraX]

Running yarn install as part of a build step for a Docker image based on node:7 fails on Travis CI with ENOTEMPTY, EEXISTS errors. It always seems to error on the webdriverio package.

yarn install v0.19.1
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/webdriverio/-/webdriverio-4.6.2.tgz: ENOENT: no such file or directory, open '/usr/local/share/.cache/yarn/npm-webdriverio-4.6.2-dd095ee618896a21c8f1b9d4278736d85a64ca0f/lib/protocol/timeouts.js'".

When Travis runs yarn install as part of the install phase it works just fine. The error only happens when building a Docker image.

Repo which reproduces this issue.

node:7
OS: Docker + Travis CI
yarn: 0.19.1
package.json
yarn.lock

I've tried installing yarn both with npm install -g and with apt and both methods cause the failure on Travis.

Weirdly enough, the image builds successfully on my local machine which runs Ubuntu 16.04.1 LTS with Docker version 1.13.0, build 49bf474.

[Daniel15]

Interesting, so it only fails on Travis, but works when testing locally? That's very weird given Docker is supposed to ensure the environment is consistent.

[NiGhTTraX]

@Daniel15 I know right...

I've downgraded node to version 6 and it still fails on Travis. I've added the --verbose flag to yarn install and all I got was

verbose Performing "GET" request to "https://registry.yarnpkg.com/spawn-wrap/-/spawn-wrap-1.3.4.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/yargs/-/yargs-6.6.0.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/yargs-parser/-/yargs-parser-4.2.1.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/fibers/-/fibers-1.0.15.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/selenium-standalone/-/selenium-standalone-5.11.2.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/tcp-port-used/-/tcp-port-used-0.1.2.tgz".
verbose Performing "GET" request to "https://registry.yarnpkg.com/babel-runtime/-/babel-runtime-5.8.38.tgz".
verbose Error: ENOTEMPTY: directory not empty, rmdir '/usr/local/share/.cache/yarn/npm-webdriverio-4.6.2-dd095ee618896a21c8f1b9d4278736d85a64ca0f/lib/protocol'
    at Error (native)
error An unexpected error occurred: "ENOTEMPTY: directory not empty, rmdir '/usr/local/share/.cache/yarn/npm-webdriverio-4.6.2-dd095ee618896a21c8f1b9d4278736d85a64ca0f/lib/protocol'".

I am open to ideas on how to debug this.

[twooster]

Downgrading to yarn 0.18.1 seemed to fix this for me. Seems like 0.19 might have a regression; see #1834

[victornoel]

I also have this problem with yarn 0.23.3, it's happening not when building an image but simply when running some CI.
The error is the following:

$ time yarn --frozen-lockfile
yarn install v0.20.3
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "ENOTEMPTY: directory not empty, rmdir '/builds/linagora/petals-cockpit/yarncache/npm-@angular/core-4.0.0-beta.8-8d9c8a64e7c26ff7208404e716deea94bb509cd7/src'".
info If you think this is a bug, please open a bug report with the information provided in "/builds/linagora/petals-cockpit/frontend/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

real    0m9.812s
user    0m7.596s
sys 0m0.932s

I think there may be some strange way of doing remove of files…

Important point: the cache was empty!

[victornoel]

And on my machine, if I try to repro, I get this:

yarn install v0.20.3
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "http://docker0.gso.lan:8081/repository/npm/@angular/core/-/core-4.0.0-beta.8.tgz: EEXIST: file already exists, mkdir '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-beta.8-8d9c8a64e7c26ff7208404e716deea94bb509cd7/src/metadata'".
info If you think this is a bug, please open a bug report with the information provided in "/home/vnoel/Linagora/Petals/dev/git/petals-cockpit-new/frontend/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[victornoel]

And with yarn 0.21.2:

yarn install v0.21.2
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "http://docker0.gso.lan:8081/repository/npm/@angular/core/-/core-4.0.0-beta.8.tgz: ENOENT: no such file or directory, lstat '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-beta.8-8d9c8a64e7c26ff7208404e716deea94bb509cd7/bundles/core.umd.js'".
info If you think this is a bug, please open a bug report with the information provided in "/home/vnoel/Linagora/Petals/dev/git/petals-cockpit-new/frontend/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

That's terrible!

[victornoel]

And I concur with @twooster about 0.18.1 working ok!

[victornoel]

@Daniel15 it doesn't work locally either. Actually, it NEVER work when the cache is empty for me!

[bestander]

@victornoel the recent error could be https://github.com/yarnpkg/yarn/issues/2714

[victornoel]

@bestander I did try 0.19.1 at the time and it didn't work…

I retried, and now the bug:

• does not appear with an empty cache, but does appear in the following case (I really hope it is reproducible…):
  
  
  
  • rm -rf the yarn cache
  
  
  • clone https://gitlab.com/linagora/petals-cockpit.git
  
  
  • checkout 5f31ccb4b2357201baa50539b30702cffceb6992
  
  
  • run yarn in the frontend directory
  
  
  • checkout master
  
  
  • run yarn again in the frontend directory
  
  
  • I get: error An unexpected error occurred: "http://docker0.gso.lan:8081/repository/npm/@angular/core/-/core-4.0.0-rc.1.tgz: ENOENT: no such file or directory, utime '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/@angular/core/testing.js'". (I am using my own registry, but the same happens without it)
  
  
• does appear with yarn 0.21.2, 0.19.1 but not with 0.18.2

So I don't think it is the same, let's hope you can reproduce it at least…

[victornoel]

(actually, I just tried again and reproduced the bug with an empty cache and yarn 0.21.2 while it wasn't the case before, maybe there is another file somewhere else that is the source of this problem, and that is not in the cache?)

[victornoel]

@bestander I will still test yarn as soon as #2744 is available as a nightly build :)

[bestander]

Ping me if I can help.
The best action is to send a PR with a broken (and skipped) e2e test.

[victornoel]

@bestander well, nope, I still get errors such as:

➜  frontend git:(master) ✗ yarn
yarn install v0.22.0-20170227.1509
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/@angular/core/-/core-4.0.0-rc.1.tgz: ENOENT: no such file or directory, lstat '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/typings/src/facade/lang.d.ts'".

or:

➜  frontend git:(master) ✗ yarn
yarn install v0.22.0-20170227.1509
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/typescript/-/typescript-2.2.1.tgz: ENOENT: no such file or directory, lstat '/home/vnoel/.cache/yarn/npm-typescript-2.2.1-4862b662b988a4c8ff691cc7969622d24db76ae9/lib/typescriptServices.js'".

I will see if I can make an e2e test.

[victornoel]

@bestander anyway I can get a complete stacktrace of the error?

I only see this in the yarn-error.log:

Trace: 
  Error: http://docker0.gso.lan:8081/repository/npm/@angular/core/-/core-4.0.0-rc.1.tgz: ENOENT: no such file or directory, lstat '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/@angular/core.es5.js'
      at Error (native)

That's a bit useless :)

[victornoel]

the detailled error is:

{ Error: http://docker0.gso.lan:8081/repository/npm/@angular/core/-/core-4.0.0-rc.1.tgz: ENOENT: no such file or directory, lstat '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/@angular/core.js'
    at Error (native)
  errno: -2,
  code: 'ENOENT',
  syscall: 'lstat',
  path: '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/@angular/core.js',
  fstream_type: 'File',
  fstream_path: '/home/vnoel/.cache/yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/@angular/core.js',
  fstream_class: 'FileWriter',
  fstream_stack: 
   [ '/home/vnoel/Linagora/Petals/dev/git/yarn/node_modules/fstream/lib/writer.js:285:28',
     '/home/vnoel/Linagora/Petals/dev/git/yarn/node_modules/graceful-fs/polyfills.js:284:29',
     'FSReqWrap.oncomplete (fs.js:123:15)' ] }

Not sure exactly what to do with this… it's happening at package-fetcher.js, line 56 exactly,  I have trouble finding the source though…

[victornoel]

It seems stupid, but I feel like it fails only when my networked npm mirror (a sonatype nexus in my company) has mirrored the @angular/core artefact. If it hasn't, things go well and then it fails on another artefact that is already mirrored (typescript in this case).

If I remove the artefact from the nexus mirror by hand, it works!

So… it is a bit like yarn can't keep up if things arrive too fast ^^ because when I use the normal npm registry without using my mirror, things usually goes well (we have a slow internet connection).
And it would explain why it often fails on CI systems because they usually have very fast internet connections…

That's a bit of stretch to conclude that, but it could help find the origin of the problem.
WDYT @bestander?

For the record, I think the error is emanating from tar.Extract step in the fetching pipeline, but I'm not totally sure ^^

[bestander]

Thanks for researching more, @victornoel, you might be on something here.

[bestander]

I can repro scenario from https://github.com/yarnpkg/yarn/issues/2629#issuecomment-282745896.
Looking into it.

I get

error An unexpected error occurred: "https://registry.yarnpkg.com/typescript/-/typescript-2.2.1.tgz: ENOENT: no such file or directory, lstat '/Users/bestander/Library/Caches/Yarn/npm-typescript-2.2.1-4862b662b988a4c8ff691cc7969622d24db76ae9/lib/typescriptServices.js'".

However if I just try yarn install again and again it will eventually finish the installation.
Looks like exploding the .tgz file ends with error.

Update:

• the .tgz seems fine, I can unzip manually the one that fails during fetch phase
• I wonder if tar package is throwing this error for some reason, could it be concurrency?

[bestander]

Some help investigating why untaring those few dependencies (in my case typescript and angular-core) causes error is welcome.
Concurrency? Bug in https://github.com/npm/node-tar?

[bestander]

@victornoel, can you reproduce the bug with yarn install --network-concurrency 1?

[victornoel]

@bestander with --network-concurrency 1 the bug does not appear (while without it, it appears every time).
But what is the default value of this parameter? Whichever value I choose for it (1, 2, 4, 8), it works, while if I don't put it at all, it fails…

[bestander]

The default is 15, I can repro the issue with concurrency 15 with a clean checkout of https://gitlab.com/linagora/petals-cockpit.git#075bac4c54fee466568c000c7ffe8025f593e212.

[victornoel]

Excellent news! One step further toward a solution AND a workaround :)

[bestander]

Some results.

TL;DR I am out of ideas how to properly fix it for good, this needs some deeper Node.js knowledge.

1. Network can be eliminated from possible issues.
   I've setup offline mirror for the .tgz files in yarn.lock and can reproduce the issue with packages installed from disk.

The issue is in unzip/untar stream in tarball-fetcher code.

1. I tried a different library that extracts tar - https://github.com/mafintosh/tar-fs vs current https://github.com/npm/node-tar/. They both fail the same way.
   Going a bit deeper - exceptions are happening in node when doing multiple mkdirp operations

Error: ENOENT: no such file or directory, chmod '/Users/bestander/Library/Caches/Yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/typings/src/di/injector.d.ts'
  errno: -2,
  code: 'ENOENT',
  syscall: 'chmod',
  path: '/Users/bestander/Library/Caches/Yarn/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3/typings/src/di/injector.d.ts' }

I think core-4.0.0 and typescript-2.2.1 fail because they have quite a few files and deep folder structures, and they fail to install while making many concurrent mkdir/copy operations.

Every time there is a different syscall that fails: chmod, rmdir, mkdir, lstat, utime.

And it is not something obvious in the libraries' code.

1. Fails the same on Node 4, 6 and 7.

2. I could not reproduce the error with concurrency set to 8, so I'll send a PR to reduce default network concurrency.

---

1. I was wondering how concurrency affects installation speed.

5.1. Using offline-mirror (no download), on my MBPro 13", clean cache and using node-tar to unzip files.
Concurrency 12 - fails
Concurrency 8 - 18 seconds
Concurrency 4 - 18 seconds
Concurrency 2 - 21 seconds

5.2. Using offline-mirror (no download), on my MBPro 13", clean cache and using tar-fs to unzip files.
Concurrency 12 - 15 seconds
Concurrency 8 - 15 seconds
Concurrency 4 - 17 seconds
Concurrency 2 - 18 seconds

5.3. Downloading packages from internet, on my MBPro 13", clean cache and using tar-fs to unzip files.
Concurrency 12 - failed once
Concurrency 8 - 21 seconds
Concurrency 4 - 23 seconds
Concurrency 2 - 34 seconds

Looks like setting concurrency to 8 is safe enough, also it makes sense to switch the tar library.
I'll follow up with a PR.

[bestander]

A proper way to fix this would be forking https://github.com/mafintosh/tar-fs and doing smarter fs operations, e.g. using mkdir for every folder only once

[victornoel]

tar-fs maintainer seems to be active, maybe we could open an issue there and see what they know/propose about this?

[bestander]

@victornoel, would you do that please?

[victornoel]

@bestander done! mafintosh/tar-fs#61 :)

[ProdigySim]

I ran into this error message in a somewhat similar scenario when testing out yarn on my jenkins' build agents.

Do you know what the conditions required to trigger this bug are? I'd like to replace my build system's npm calls with yarn for speed, but if I have to disable concurrency I'm worried it might negate any bonus there.

[victornoel]

@ProdigySim, as explained in #2829 (which was merged in yarn master), reducing the network concurrency has no big effect on the performance of yarn. You can simply set it to 8 and it should be alright. Anyway, even when downloading 8 dependencies at a time, I'm not sure most drive is going to follow in throughput so you won't loose much for sure :)

[ProdigySim]

@victornoel Thanks for the information. I'm not sure just reducing --network-concurrency will be enough in my case, since we would also run multiple instances of yarn in parallel.

I can repro this issue even with --network-concurrency 1, but maybe I should open a separate issue for that?

Using the same test repo you gave above:

#!/bin/bash
set -x # echo commands

# Clear yarn cache
rm -rf $(yarn cache dir)

# Clone the repo into two separate spots
git clone https://gitlab.com/linagora/petals-cockpit.git repo1
git clone https://gitlab.com/linagora/petals-cockpit.git repo2

# Run yarn on both in parallel
cd repo1/frontend && yarn --network-concurrency 1 &
cd repo2/frontend && yarn --network-concurrency 1 &

This nets me the error every time (4 for 4 so far)

error An unexpected error occurred: "https://registry.yarnpkg.com/@angular/core/-/core-4.0.0-rc.2.tgz: 
ENOENT: no such file or directory, lstat '/Users/<snip>/Library/Caches/Yarn/npm-@angular/core-4.0.0-rc.2-59535050e5d0e6141417186eee571296f8e9c3d0/@angular/core.es5.js'".

On yarn 0.21.3, node v4.5.0, OSX 10.11.6

So far I've been able to work around this issue by only including yarn on build jobs that will never run in parallel, or use completely different sets of packages, but even then I'm not sure it will be safe--hence asking about root conditions for this bug.

[jiaweihli]

@ProdigySim
This is a separate, but related, issue caused by Yarn's global download cache. A workaround is to create a different cache for each directory.

You can still run with --network-concurrency 8. (I actually have no issues with unlimited network concurrency.)

More context here.

[victornoel]

@bestander surprisingly, today, the problem reappeared (triggered by a tar for a new version of angular ^^) even with the network concurrency at 8, but only on my CI… I moved it to 2 and it works (and I don't really care if it takes a few more seconds to download dependencies, so it's ok for now).
We don't seem to get feedbacks from the tar-fs project… who else could we contact for help on that?

[kevingelion]

having this issue as well on my Travis builds for OS X. I've clearned the cache and set network concurrency but nothing has helped.

[victornoel]

@kevingelion to which value did you set the network concurrency? be drastic and set something like 2 to see if the problem is this one :)

[kevingelion]

@victornoel I set it to 1 and 2, and both options resulted in a fail. I used yarn --mutex network and no dice either.

[victornoel]

@bestander the following hack fixes (edit: NOT) the problem:

diff --git a/src/util/request-manager.js b/src/util/request-manager.js
index e0e134a2..995dac69 100644
--- a/src/util/request-manager.js
+++ b/src/util/request-manager.js
@@ -214,8 +214,7 @@ export default class RequestManager {
     }, params.headers);

     const promise = new Promise((resolve, reject) => {
-      this.queue.push({params, resolve, reject});
-      this.shiftQueue();
+      this.execute({params, resolve, reject});
     });

     // we can't cache a request with a processor

Obviously it is not a fix, it completely bypasses the request manager and its queuing system, but it shows that the problem is coming from this subsystem.

[bestander]

Thanks, Victor!

On 24 March 2017 at 18:07, Victor Noël [e-mail] wrote:

@bestander https://github.com/bestander the following hack fixes the
problem:

diff --git a/src/util/request-manager.js b/src/util/request-manager.js
index e0e134a2..995dac69 100644
--- a/src/util/request-manager.js
+++ b/src/util/request-manager.js
@@ -214,8 +214,7 @@ export default class RequestManager {
}, params.headers);

 const promise = new Promise((resolve, reject) => {

• this.queue.push({params, resolve, reject});
• this.shiftQueue();
• this.execute({params, resolve, reject});
  });

 // we can't cache a request with a processor

Obviously it is not a fix, it completely bypasses the request manager and
its queuing system, but it shows that the problem is coming from this
subsystem.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/2629#issuecomment-289102067, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWF66L-NzAInx7Bhs6V7s7LKahxxUks5rpAZ1gaJpZM4L3JbX
.

[victornoel]

whoups not, it doesn't :D but it does improve things a bit

[victornoel]

Sorry for the false positive, I was too eager to report my findings :)

It changes things because before I could run yarn many times and it would never succeed downloading the angular-core dependency or the typescript (always these ones), but there it failed the first time, and succeeded the second time, and I forgot to remove the cache in between my tries so I thought it was working.

[victornoel]

Well it depends, now sometimes it works, sometimes it does not (with the hack, so it's not a total miss or my internet connection is too slow right now...)

[kaicataldo]

I'm running into this too in our CI builds. After a lot of testing, I've also finally been able to reproduce locally.

Again, it does sometimes work, but it fails often with either of the following errors (which makes it seem like there's some kind of race condition somewhere):

• ENOENT: no such file or directory, lstat 'cache/directory/some-file'
• EEXIST: file already exists, mkdir 'package-name'

I've isolated it to one package that we install directly from a private repository on GitHub. Interestingly enough, the package being referenced in the error message is always a dependency of this package (and it is always another package we install directly from GitHub, though not a private repository). So it seems like one repro case is installing packages from private GitHub URLs who have subdependencies that are also installed from GitHub repositories (not necessarily private).

Not sure if this helps at all...I'm happy to try to help out in any way I can!

Edit: Not sure if this is helpful or not, but the top level package is listed in the format of "git+ssh://[email protected]/org/package.git#v1.0.0", and in the error, the subdependency being downloaded looks like it's being downloaded over https with a url of "https://codeload.github.com/org/package/tar.gz/ljasdf08i234098aifj".

[bestander]

I am investigating this a bit more.
Trying to build a standalone script for concurrent tar-fs extracts but I tend to think that the problem is in tar file breaking during download.

[bestander]

Found it, Doh.

In the example from https://github.com/yarnpkg/yarn/issues/2629#issuecomment-282745896 Yarn has duplicate packages that are getting downloaded and extracted in parallel.
The duplicated ones are @angular/core/-/core-4.0.0-rc.1 and typescript/-/typescript-2.2.1.tgz.

With high concurrency we just happen to do concurrent extraction into the same cache folder.
I'll investigate why Yarn does not deduplicate those two pacakages and send a fix.

No magic on OS or tar extraction level.

[victornoel]

haha, good job @bestander, glad we finally found the problem!

[kaicataldo]

Awesome work @bestander :tada:! Running into both https://github.com/yarnpkg/yarn/pull/3090 and https://github.com/yarnpkg/yarn/pull/3106 was what blocked us from using Yarn.

[victornoel]

thanks!

[skylize]

I had this problem installing prop-types module. Each time I tried to install it would ENOENT on a different filename. For me the problem went away after installing npm 5.0.2

$ yarn add prop-types
yarn add v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/prop-types/-/prop-types-15.5.10.tgz: ENOENT: no such file or directory
....

$ npm -g install npm

# whoops, looks like npm installed itself to different location than apt-get did
$ npm -v 
3.5.2

# remove the cached link from shell so the right version can surface
$ hash -d npm
$ npm -v
5.0.2

$ yarn add prop-types
... properly installs prop-types as expected

[Daniel15]

@skylize That's likely to be coincidental - Yarn doesn't use the npm client at all, so the version of npm does not affect execution of Yarn at all.

[Redmega]

This is causing my Travis builds to fail almost every single time, with a few different packages. Is there a solution yet?
error An unexpected error occurred: "https://registry.yarnpkg.com/apollo-client/-/apollo-client-1.8.0.tgz: ENOENT: no such file or directory, utime '/var/lib/jenkins/.cache/yarn/v1/npm-apollo-client-1.8.0-3b5d1976a06a0f82b2fc66fe71754868193dadb9/flow-typed/npm/webpack_vx.x.x.js'".

[benmerckx]

@Redmega
Same here, but this works:

yarn install --network-concurrency 1

[victornoel]

Which version are you using? This is meant to be fixed already...

Le 8 août 2017 6:37 PM, "Ben Merckx" [e-mail] a écrit :

@Redmega https://github.com/redmega
Same here, but this works:

yarn install --network-concurrency 1

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/2629#issuecomment-321011749, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAJ0z5qFb7gSW4w14_RbFNjsn4sRYV78ks5sWI7hgaJpZM4L3JbX
.

[Redmega]

@victornoel I am using v0.27.5 on the jenkins machine, same as my local.

[BYK]

Please try the nightlies: https://yarnpkg.com/en/docs/nightly

[headione]

Removing the yarn.lock file and yarn install fixed the problem for me.

[ajcrites]

This also causes my Jenkins builds to fail occasionally. Usually it works after a second try but will fail again later.

[victornoel]

@ajcrites @Redmega @headione @benmerckx you should open another issue if you are experiencing this kind of problem. This issue has been fixed for sure, so your problem must be a different one, even if it shows some similar symptoms.
I'm pretty sure there is more chance for your problem to be solved if you open another issue :)

[floric]

We have the same problem, doing parallel builds of packages in Jenkins with Node 8.5. We currently need to stick to 0.27.5, until 1.0.2 is released fixing another bug. But thanks for your support and work anyway :)

[Niceplace]

@floric I'm getting the same issue with the same context (Jenkins + Parallel) with node 8.9.4, has your issue been resolved ?

Edit: I'll try to use 8.11.1 to see if it includes a latest version of yarn without the bug.

[BYK]

@Niceplace you may wanna try the --mutex option: https://yarnpkg.com/en/docs/cli#toc-concurrency-and-mutex

We have plans for adding better per-package locking to avoid this soon.

[FezVrasta]

I'm having intermittent errors with both ENOENT: no such file or directory, chmod and ENOENT: no such file or directory, lstat trying to run yarn --mutex=network on the root of a monorepo with Yarn workspace enabled...

It doesn't seem to be consistent, I get either one or the other randomly. (1.6.0 and node 8.11.1 and 9.11.1)

Specifically, the errors are:

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/Users/federicozivolo/test/packages/foobar/node_modules/detect-port-alt'".

and

error An unexpected error occurred: "ENOENT: no such file or directory, chmod '/Users/federicozivolo/test/packages/foobar/node_modules/jest/node_modules/.bin/jest'".

[nieltg]

I'm running Yarn 1.7.0 and I'm having the similar error. Yarn finally manages to install the package after several runs.

An unexpected error occurred: "ENOENT: no such file or directory, lstat '/home/nieltg/.cache/yarn/v1/npm-npm-registry-client-8.5.1-8115809c0a4b40938b8a109b8ea74d26c6f5d7f1/lib/dist-tags/fetch.js'".

EDIT:
I've used yarn --network-concurrency 1 but the error still occurs on me. Here is another sample of the error and yarn-error.log file.

An unexpected error occurred: "ENOENT: no such file or directory, copyfile '/home/nieltg/.cache/yarn/v1/npm-core-js-2.5.7-f972608ff0cead68b841a16a932d0b183791814e/library/fn/date/now.js' -> '/mnt/c/Users/nieltg/Projects/React/React-16-Demo/node_modules/core-js/library/fn/date/now.js'".

[rdsedmundo]

I'm using Yarn 1.7.0. And I can confirm the same behavior still happening to me.

It's completely random. Sometimes it happens, sometimes it doesn't.

The last I received was:

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/root/.yarn-cache/v1/npm-@storybook/addon-actions-3.4.5-ba0d0c0c74357c0852e0b890b40

[ghost]

I'm seeing this error quite frequently with Yarn 1.9.2 on Windows Subsystem for Linux.

[antonovicha]

Today we got similar problems with broken packages on Jenkins CI where pipeline run yarn install in parallel. It was working fine jest few days ago.
Fixed with yarn install --network-concurrency 1 (as mentioned in a comment). Performance did not degraded much: ~7 sec -> ~8 sec.

[tommedema]

Why was this closed? It still happens:

Toms-MacBook-Pro-2:design-to-code tommedema$ yarn install
yarn install v1.9.4
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/Users/tommedema/projects/vg/design-to-code/packages/vgcli/node_modules/fs-extra'".
info If you think this is a bug, please open a bug report with the information provided in "/Users/tommedema/projects/vg/design-to-code/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Toms-MacBook-Pro-2:design-to-code tommedema$ yarn install --network-concurrency 1
yarn install v1.9.4
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
✨  Done in 24.85s.

Note that in my case it disappeared after doing yarn remove fs-extra and yarn add fs-extra in two packages, effectively upgrading this dependency.

[nieltg]

Hi, I think I found something.

I was dabbling with a piece of code which list files in a specified directory recursively using fs and rxjs and I discovered that it was likely to fail if I don't wait for lstat call to be finished before calling another lstat.

I've made a simple NPM package, async-dirtree-test, to test if your environment is affected or not with this. I'm using WSL and I found that it is likely to fail while handling a directory which has many child directories, like node_modules, even with a low number of concurrency.

Well, I still don't know if this problem is WSL specific or not. Right now I can't test it in another environment, like Linux, Mac, etc.

[rainabba]

@nieltg I wanted to share an observation that might help shape some of the others. I'm using Docker CE in WSL and Docker for Windows as my host so when I work with Docker in WSL, it feels native, but in reality the host operates in the Windows world natively (so my Dockerfiles actually resolve /c/foobar to c:/foobar in the Docker engine). This is incredibly relevant when I use bindings (inside my container, I'm mounting my local folders so that /usr/src inside the container is ultimately at c:/src/foobar (though my Dockerfile would show the binding as /c/src/foobar:/usr/src (see the automatic translation in the path?)

This distinction is important because if I yarn install inside one of those local folders, inside my container, I get the same errors I do directly in WSL (no Docker involved).

On the other hand.... if I just mkdir /tmp/src && cp ./package.json /tmp/src/ && cd /tmp/src && yarn install, everything succeeds perfectly and I can just mv /tmp/src/node_modules /c/src/foobar/ and I'm good, so that's my present workaround. Mind you that /tmp exists as a docker store (all the IO looks like a single file to the OS because it's effectively a partition in a file).

I know that involving docker isn't ideal here, but it seems to suggest that rapid file handles might be an issue since IO itself isn't and might help others work around this.

...got distracted and submitted too soon. Anyhow, my brain is elsewhere at the moment, but I'll come back later and see if I can devise a test using your approach and Docker to draw further conclusions.

[konradjurk]

REOPEN

[Marclev78]

We have recently started seeing the same sort of error using yarn 1.10.1, running a CI build in Azure Devops (formerly Visual Studio Team Services).

The actual dependency that is failing seems to be random best I can tell, but yarn install is falling over intermittently with the ENOENT: no such file or directory, open '/usr/local/share/.cache/yarn........ error. One time it the build will work, the next it will fail.

The workaround of yarn install --network-concurrency 1 seems to work for us.

[malware27]

@Marclev78 same error but yarn install --network-concurrency 1 doesn't seem to work for me

[Fabianopb]

@Marclev78 same thing here, using yarn 1.10.1 in Azure Devops and getting the error:

Error: https://registry.yarnpkg.com/core-js/-/core-js-1.2.7.tgz: ENOENT: no such file or directory, utime 'C:\Users\grpsshagent\AppData\Local\Yarn\Cache\v1\npm-core-js-1.2.7-652294c14651db28fa93bd2d5ff2983a4f08c636\fn\string\pad-left.js'

Locally everything works as expected.

[jsatk]

I'm here to simply say I too am seeing this error.

error An unexpected error occurred: "ENOENT: no such file or directory, chmod '/usr/local/opt/asdf/installs/nodejs/8.12.0/.npm/bin/atob'".

Unfortunately I think I have to abandoning yarn for my global node binaries and moving back to npm until this is fixed.

[Nopik]

Sadly, this problem started to plague our CI builds recently, too ;(

[error-bruno]

@rainabba suggestion worked for me on WSL

[rainabba]

I think that write and read operations behave differently also. Even with my hacks, I frequently see errors while calling node's fs.writeFile (wrapped with bluebird promisify though). In every single instance, I can confirm the file exists immediately after I get the write error.

I'm sending a string (XML file content) to fs.writeFile(), which ultimately calls the following, but I'm not sure if I'm prepared to undertake the challenge that be required to setup a custom build with additional debugging output from this C++ project, so that I can confirm exactly where the right is actually feeling according to node or this C++ module.

Bottom line is that the writes are not failing, but node believes they are so the scenario that makes sense to me is that the c+ plus module is succeeding, but internally it checks for the file and fails, then reports not feel your back to node and then the actual write occurs so that when I go to check for the file, it is there and the error makes no sense.

https://github.com/nodejs/node/blob/master/src/node_file.cc#L1795

[victornoel]

@bestander any way to get that issue reopened? Clearly this is not fixed and impacting a lot of people.

[arichiardi]

Confirming this still happens with yarn 1.12 and Azure Pipelines.

[bestander]

Thanks for confirming everyone.
Looks like there are multiple reasons for that error.
I'll reopen the issue, community help to debug this is needed though.

[TTouka]

also happens with yarn 1.11, but not with 1.10

[jhalborg]

@bestander - Related? https://github.com/yarnpkg/yarn/issues/6312

If so, there's some fine repro work inthere

[limonte]

I'm affected by this issue as well.

Windows 10 / WSL

"ENOENT: no such file or directory, lstat '/mnt/c/Users/<username>/.cache/yarn/v4/<random_file_in_random_package>"

[rafsawicki]

@limonte WSL had an error for a while that it would randomly throw a similar error when executing npm install / yarn install. It was happening when a lot of files at once were copied to the hard drive. Please make sure that you're on the latest Windows version (1809 or higher), as it might not be caused by the yarn itself.

[mobilutz]

We are also seeing the problem of Extracting tar content of undefined.

error https://registry.yarnpkg.com/eslint/-/eslint-4.19.1.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat '/tmp/yarncache.KTKNZ/v4/npm-eslint-4.19.1-32d1d653e1d90408854bfb296f076ec7e186a300/node_modules/eslint/lib/rules/no-compare-neg-zero.js'"

So far we mitigated this with using only one concurrent network connections with the option --network-concurrency 1. But this is more a temporary solution.

[krns]

I also can confirm the issue on node:11.5.0-alpine.

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/app/node_modules/<random_pacakge>

[cinderblock]

I noticed that the problems seemed to be related to linking to the git repository version of packages.

Reproduce

package.json

{
  "dependencies": {
    "react-navigation-core": "https://github.com/react-navigation/react-navigation-core",
    "react-navigation-hooks": "https://github.com/react-navigation/react-navigation-hooks"
  }
}

rm -rf node_modules && yarn cache clean && yarn

Workaround

Setting network-concurrency 1 fixes the problem for me every time.

Running npm install also works.

Notes

Removing either package from the dependency list doesn't cause the error, nor does using the published npm versions of those packages.

It seems to throw a different error every time. It seems to happen randomly to different files and with slightly different errors.

error https://registry.yarnpkg.com/core-js/-/core-js-1.2.7.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod  '/home/cameron/.cache/yarn/v4/npm-core-js-1.2.7-652294c14651db28fa93bd2d5ff2983a4f08c636/node_modules/core-js/library/modules/es6.reflect.apply.js'"

Variations

• ENOENT: no such file or directory, chmod
• ENOENT: no such file or directory, stat
• ENOENT: no such file or directory, open
• EEXIST: file already exists, mkdir

Other messages

info There appears to be trouble with your network connection. Retrying...

[Nopik]

Indeed, I've just tried to reproduce using your package.json and error showed up on first try.

[rainabba]

What layer does the WSL filesystem interact with the NTFS fs that's already in place?

Are you folks seeing this error in a mounted drive (/c or /mnt/c for common examples), or outside one of those mounts? Mind testing aan alternate (~/. for example) and reporting any difference?

My intuition is nagging at me, but I may be confusing issues with my docker experiences and I need to check this independently.

[freemanercom]

[2/4] Fetching packages...
error https://registry.yarnpkg.com/smartwrap/-/smartwrap-1.0.10.tgz: Extracting tar content of undefined failed, the file appears to be co
rrupt: "ENOENT: no such file or directory, open 'C:\Users\Administrator\AppData\Local\Yarn\Cache\v4\npm-smartwrap-1.0.10-873ef350d
4ee1262fed4a80a55634d86ae1faf48\node_modules\smartwrap\ejq'"
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.

[w-biggs]

Are you folks seeing this error in a mounted drive (/c or /mnt/c for common examples), or outside one of those mounts? Mind testing aan alternate (~/. for example) and reporting any difference?

Is there a consistently reproducible case where this is happening? It's been pretty random for me, but I do all of my yarn add-ing on a mounted drive and this occurs frequently.

I was able to reproduce https://github.com/yarnpkg/yarn/issues/2629#issuecomment-451638917 in both a mounted drive and in ~.

I also tried to reproduce https://github.com/yarnpkg/yarn/issues/2629#issuecomment-282745896 but it kept failing to fetching the last package, which I'm pretty sure is unrelated.

[lizozom]

I had the same issue in the last few hours. yarn was randomly failing to install various packages, showing the errors that were mentioned above.

I tried reseting yarn cache and re-installing and running with network-concurrency 1, neither of which had worked.

What solved the problem for me, was switching to a different network (just used my phone AP instead of WiFi) and everything worked like magic.

I have a hunch that this issue might be related to mishandled recovery for some very specific network errors. Will look into it later.

[mucsi96]

I can confirm the previous comment. Setting network-concurrency doesn't help. Switching to Phone hotspot fixed the issue for me. My environment: Windows 10 (Linux subsystem - Ubuntu)

[sauntimo]

I'm on WSL and had been seeing this issue with the geo-tz package which has a (slightly odd) deeply nested folder structure. I tried some --network-timeout and --network-concurrency things but got nowhere. However, when I enabled long paths on Windows (see this SuperUser post) it's now working fine. Perhaps this might help some other people on WSL. Edit it looks like I spoke too soon. It was working, and linking dependencies is faster, but now I'm seeing the same error again.

[kunokdev]

Still breaking CI....

[Hintalo]

We have the same problem with Yarn 1.13.0 running on a Debian Linux machine that serves as a Jenkins slave node. Note that we have a local yarn repository server, so during the build, there is no (or very few) physical downloads from public-internet repo servers.

yarn install v1.13.0
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "http://sqrep01.rsint.net:4873/lodash/-/lodash-4.17.10.tgz: ENOENT: no such file or directory, open '/home/jenkins/.cache/yarn/v4/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7/node_modules/lodash/.yarn-tarball.tgz'".

The actual file is existing, both on our repo server and on the file-system.
If we start the build again, the build may be successful or may fail with some other (random) file.
We didn't alter the default network-concurrency setting.

[jkonowitch]

Ditto - This is still an issue, even in 1.14

Arguments: 
  /home/jeff/n/bin/node /usr/share/yarn/bin/yarn.js install

PATH: 
  /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/Program Files/Git/cmd:/mnt/c/Users/jkono/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/jkono/AppData/Local/hyper/app-2.1.2/resources/bin:/mnt/c/Users/jkono/AppData/Local/Programs/Microsoft VS Code/bin:/home/jeff/n/bin

Yarn version: 
  1.14.0

Node version: 
  10.15.1

Platform: 
  linux x64

Trace: 
  Error: ENOENT: no such file or directory, scandir '/mnt/c/Users/jkono/dev/PROJECT/node_modules/@storybook/addon-links/src'

Also:

➜  yarn cache dir
/mnt/c/Users/jkono/home/.cache/yarn/v4

[jwaldrip]

This is pretty annoying, we see this daily on local and ci machines.

[arichiardi]

Confirming this is happening all the time in CI for us as well

[entersb]

Hi, confirming that this issue happens on our CI.
This is line with problem.

error https://registry.yarnpkg.com/core-js/-/core-js-1.2.7.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v4/npm-core-js-1.2.7-652294c14651db28fa93bd2d5ff2983a4f08c636/node_modules/core-js/es7/regexp.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[FezVrasta]

The same issue started happening today to one of our open source projects.

You can see a failing build here:
https://travis-ci.com/quid/refraction/builds/103692106

And one that succeeds (with --network-concurrency 1) here:
https://travis-ci.com/quid/refraction/builds/103693682

I hope it can help to diagnose the issue!

The source code of the repository is at:
https://github.com/quid/refraction

[Hintalo]

Maybe this helps somebody:
On our Jenkins CI, the problem was that Jenkins triggers parallel builds for our apps, meaning that two (or more) shell scripts have triggered "yarn install" at the same time, where one of the build processes has further _removed the yarn-cache completely_ (using "yarn cache clean") before starting "yarn install". This was a fatal issue for the other yarn processes of course.
We then removed the cache-cleaning and changed the yarn command to
yarn install --verbose --prefer-offline --mutex file:/tmp/.yarn-mutex --network-concurrency 1
(_--verbose_ is not really necessary) and inserted
child-concurrency 1
in .yarnrc.
Now, as the parallel builds are triggered, yarn has detected that another yarn process is active and waited until it finishes. This solved the "no such file" problems on our CI.

[jeremyjs]

I get this issue on my local machine whenever I use a package reference with this format:

"connect-js-adapter-tls": "git+https://github.com/jeremyjs/connect-js-adapter-tls.git#v3.2.2",

Notable charactaristics: private package, github url, git+https, tagged git reference

Steps which reproduce for me:

1. Clean slate: Remove all such references and run yarn install. It works fine.
2. Add all such references back to my package.json and run yarn install again, it still works fine on the first run after these references are added back.
3. Running yarn install additional times after that works as long as there are no changes.
4. However, modify any packages and run yarn install and I get the error.
5. If I then remove all such packages and run yarn install I do not get the error. That brings us back to step 1.

Error looks like this:

error An unexpected error occurred: "ENOENT: no such file or directory, open '/Users/jeremy/Library/Caches/Yarn/v4/npm-connect-js-adapter-tls-3.2.2-0c97726d92c21183a7fb7334344eb5047e8bc158/node_modules/connect-js-adapter-tls/.yarn-metadata.json'".

If I remove all git tag references, I observe the same behavior. So I believe that can't be the issue.
i.e.

"connect-js-adapter-tls": "git+https://github.com/jeremyjs/connect-js-adapter-tls.git",

Running npm install also gives an error:

npm ERR! premature close

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jeremy/.npm/_logs/2019-03-20T04_38_38_739Z-debug.log

npm-debug.log: https://gist.github.com/jeremyjs/e97381b16f46124ff7a9bd75ad79fd62

[jeremyjs]

As a follow up, I used a workaround of making a package.json script to clean those packages from my cache before installing:

"install-clean": "yarn cache clean connect-js-adapter-tls connect-js-api connect-js-codec connect-js-encode-decode connect-protobuf-messages && yarn install"

[iDVB]

Is there any idea yet what this issue could be?
In our case, we are running yarn install as part of our CI (inside docker container) and getting these same errors. We've tried yarn cache clean

Not sure what else to try at this point and its halting our builds. 😬

[tevaum]

Dan, did you try running with --network-concurrency 1 ? I have a similar
scenario and that solved my problem.
On Apr 2, 2019 22:17, "Dan Van Brunt" [e-mail] wrote:

Is there any idea yet what this issue could be?
In our case, we are running yarn install as part of our CI (inside docker
container) and getting these same errors. We've tried yarn cache clean

Not sure what else to try at this point and its halting our builds. 😬

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/2629#issuecomment-479283590, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFU4O1iKA-HBd62Hema1ETmuUlMro_GLks5vdAEOgaJpZM4L3JbX
.

[kunokdev]

@tevaum that solved problem for our CI as well. It also slowed down our builds significantly. So terrible, yet only workaround.

[tevaum]

Yeah. It's a bad downside. You can try with small numbers like 2 or 4... it
will be a little faster, but for me, the only value that worked was 1 :/

So we'll have to wait for the real fix to be happy ;)
On Apr 4, 2019 00:26, "kunokdev" [e-mail] wrote:

@tevaum https://github.com/tevaum that solved problem for our CI as well.
It also slowed down our builds significantly. So terrible.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/2629#issuecomment-479735791, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFU4O1a9lHn41K0eEQT9zZZzOoATiT61ks5vdXD8gaJpZM4L3JbX
.

[sarink]

Will this ever be fixed? This has rendered yarn unusable on more than one of my projects

[BYK]

Please use the mutex option, documented here: https://yarnpkg.com/en/docs/cli/#toc-concurrency-and-mutex

Yarn's cache usage is not safe to use in a multi-process manner and this is the reason for such errors.

Alternatively, you can set a per-process cache folder using the --cache-folder option documented here: https://yarnpkg.com/en/docs/cli/cache#change-the-cache-path-for-yarn-

The downside of using the mutex option is having only one yarn instance in the entire machine (others will wait until the active process terminates) which means no concurrency at all across CI jobs.

The downside of a per process cache folder is increased I/O and some lost potential due to less cache reuse.

The ideal solution would be to implement a process-safe cache implementation which is not very easy with the lack of any file locking capabilities in Node (the only reliable option seems to be creating mutex directories). The second best is to use a per-parallel-arm cache folder which would allow concurrency and cache reuse in the same arm.

[Nopik]

I don't think it is the mutex stuff. Me and some others do run single yarn at a time - in my case, it is just RUN yarn install in a Dockerfile - during docker image build phase. That makes it pretty sure that no other processes are running simultaneously in that environment.

[Nopik]

Check this out, minimal reproduction example (at least for my OSX):

728 22:49:55 iMac ~/tmp/ynse$ ls
Dockerfile  package.json
729 22:49:58 iMac ~/tmp/ynse$ cat Dockerfile
FROM node
ADD . /app
WORKDIR /app
RUN yarn

730 22:50:00 iMac ~/tmp/ynse$ cat package.json
{
  "dependencies": {
    "react-navigation-core": "https://github.com/react-navigation/react-navigation-core",
    "react-navigation-hooks": "https://github.com/react-navigation/react-navigation-hooks"
  }
}
731 22:50:03 iMac ~/tmp/ynse$ docker build -t yt .
Sending build context to Docker daemon  15.87kB
Step 1/4 : FROM node
 ---> 39337023f8d4
Step 2/4 : ADD . /app
 ---> aa86b2d7f191
Step 3/4 : WORKDIR /app
 ---> Running in 83baa8603935
Removing intermediate container 83baa8603935
 ---> 80741f170292
Step 4/4 : RUN yarn
 ---> Running in 0718118bdcd6
yarn install v1.3.2
warning package.json: No license field
info No lockfile found.
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/app/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error An unexpected error occurred: "https://registry.yarnpkg.com/lodash/-/lodash-4.17.11.tgz: EEXIST: file already exists, mkdir '/usr/local/share/.cache/yarn/v1/npm-lodash-4.17.11-b39ea6229ef607ecd89e2c8df12536891cac9b8d'".
^C
732 22:50:23 iMac ~/tmp/ynse$

[BYK]

@nopik - Yarn 1.3.2 is very old and there were numerous fixes after that version. Have you tried using one of the latest inside Docker?

[Nopik]

Indeed, that node image was quite old. Here is fresh one downloaded few minutes ago from dockerhub node:latest:

Sending build context to Docker daemon  15.87kB
Step 1/4 : FROM node
 ---> a9c1445cbd52
Step 2/4 : ADD . /app
 ---> Using cache
 ---> 26ed37136c09
Step 3/4 : WORKDIR /app
 ---> Using cache
 ---> b2339e7d25af
Step 4/4 : RUN yarn
 ---> Running in cdbdfd9c373c
yarn install v1.15.2
warning package.json: No license field
info No lockfile found.
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "ENOTEMPTY: directory not empty, rmdir '/usr/local/share/.cache/yarn/v4/npm-lodash-4.17.11-b39ea6229ef607ecd89e2c8df12536891cac9b8d/node_modules/lodash'".
info If you think this is a bug, please open a bug report with the information provided in "/app/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[Nopik]

@BYK Have you tried to build this docker yourself?

[BYK]

Will try in a few days once I get to my computer (on mobile now). Looks very weird but the directory is consistent. I'll try to see what's going on, thanks for the repro case.

[BYK]

@nopik - looking closer at the log, it indeed shows multiple Yarn instances running in parallel. You should never see the same "Resolving packages" message twice. I don't know why that's happening but I'm 100% sure that's the reason.

[Nopik]

@BYK I do see, that one of those packages has "scripts": { "build": "yarn babel --out-dir dist && del-cli 'dist/**/__tests__' && yarn tsc --emitDeclarationOnly", "prepare": "yarn build" } the other has different scripts, but still running yarn in prepare.. Are those launched by yarn during install?

[BYK]

@Nopik - don't think these would cause the issue as they are simply running scripts, not another install. Also those scripts are run after the I/O phase. There must be something else triggering multiple yarn install instanaces.

[cinderblock]

I agree that it's probably multiple instances of yarn running concurrently. The problem is that this happens on a single cli invocation of yarn. This does not need docker to reproduce.

My, admittedly underinformed, theory is that it's something related to the prepare step and that yarn might be launching an extra instance to "build" git packages. In particular I bet it is two dependencies that each depend on a common package that also needs to be built. Yarn tries to be smart and build each separate package in parallel but fails when it tries to build two packages in the same cache location.

[entersb]

In our case, we do not have multiple yarn instances.

Our system runs in its own docker image. It has single _yarn install_. It started misbehaving all of sudden and now we cannot work without network concurrency set to 1.
Unless yarn itself changed over night, I do not see issue other than that yarn is making issue itself with specific conditions.

[sarink]

If you try to solve this issue with --mutex file or even --mutex network, you'll inevitably (probably) just run into this bug https://github.com/yarnpkg/yarn/issues/6650 (open/unresolved for 6 months now) 😢

Which means once you run yarn install, although it will succeed, you'll never be able to execute another yarn command

[iDVB]

Dan, did you try running with --network-concurrency 1 ? I have a similar scenario and that solved my problem.

@tevaum - That was exactly the issue. THANKS!
I had a script that I did not think would ever be running more then one instance but it was. 🤦‍♂️

[MatthewGreenlaw]

@tevaum solved for me too. Thank you.

[illarionvk]

If you try to solve this issue with --mutex file or even --mutex network, you'll inevitably (probably) just run into this bug #6650 (open/unresolved for 6 months now)

@sarink — I ran into the mutex option bug too; thank you for mentioning it.

[cinderblock]

yarn says everything is fine.

PS C:\Users\chtacklind\Desktop\git\Project> yarn --verbose
yarn install v1.10.1
verbose 0.282 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.284 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.285 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 0.286 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc".
verbose 0.288 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.289 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.29 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\.npmrc".
verbose 0.291 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\.npmrc".
verbose 0.295 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 0.297 Checking for configuration file "C:\\Users\\.npmrc".
verbose 0.3 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.301 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.302 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.309 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.312 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\yarnrc".
verbose 0.317 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.318 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.319 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\.yarnrc".
verbose 0.326 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\.yarnrc".
verbose 0.327 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.333 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.336 Checking for configuration file "C:\\Users\\.yarnrc".
verbose 0.346 current time: 2019-05-12T11:56:12.800Z
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.33s.

Yet running yarn --check tries to build but always fails.

Sometimes with garbled error messages at the end:

PS C:\Users\chtacklind\Desktop\git\Project> yarn --check-files --network-concurrency 1 --mutex file:C:/.yarn-mutex --verbose
yarn install v1.10.1
verbose 0.286 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.288 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.289 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 0.29 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc".
verbose 0.291 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.292 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.npmrc".
verbose 0.293 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\.npmrc".
verbose 0.294 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\.npmrc".
verbose 0.295 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 0.296 Checking for configuration file "C:\\Users\\.npmrc".
verbose 0.302 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.304 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.305 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.306 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.307 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\yarnrc".
verbose 0.308 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.311 Found configuration file "C:\\Users\\chtacklind\\Desktop\\git\\Project\\.yarnrc".
verbose 0.313 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\git\\.yarnrc".
verbose 0.314 Checking for configuration file "C:\\Users\\chtacklind\\Desktop\\.yarnrc".
verbose 0.315 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.316 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 0.317 Checking for configuration file "C:\\Users\\.yarnrc".
verbose 0.32 current time: 2019-05-12T11:56:20.033Z
[1/4] Resolving packages...
[2/4] Fetching packages...
verbose 2.344 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\ef8122f161347726dd1763e1dca6eeef.d34344cbdf2ff518a03c08fc5f46827c9d66e543.prepare\\.npmrc".
verbose 2.344 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 2.345 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc".
verbose 2.345 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\ef8122f161347726dd1763e1dca6eeef.d34344cbdf2ff518a03c08fc5f46827c9d66e543.prepare\\.npmrc".
verbose 2.346 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\.npmrc".
verbose 2.346 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.npmrc".
verbose 2.346 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.npmrc".
verbose 2.346 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\.npmrc".
verbose 2.347 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\.npmrc".
verbose 2.347 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\.npmrc".
verbose 2.347 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 2.348 Checking for configuration file "C:\\Users\\.npmrc".
verbose 2.348 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\ef8122f161347726dd1763e1dca6eeef.d34344cbdf2ff518a03c08fc5f46827c9d66e543.prepare\\.yarnrc".
verbose 2.349 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 2.35 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 2.351 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\yarnrc".
verbose 2.352 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\ef8122f161347726dd1763e1dca6eeef.d34344cbdf2ff518a03c08fc5f46827c9d66e543.prepare\\.yarnrc".
verbose 2.353 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\.yarnrc".
verbose 2.358 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.yarnrc".
verbose 2.359 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.yarnrc".
verbose 2.36 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\.yarnrc".
verbose 2.361 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\.yarnrc".
verbose 2.362 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\.yarnrc".
verbose 2.363 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 2.364 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 2.366 Checking for configuration file "C:\\Users\\.yarnrc".
[1/4] Resolving packages...
[2/4] Fetching packages...
verbose 2.541 Performing "GET" request to "https://registry.yarnpkg.com/typescript/-/typescript-3.3.3333.tgz".
verbose 3.263 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.npmrc".
verbose 3.264 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 3.265 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc".
verbose 3.266 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.npmrc".
verbose 3.268 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\.npmrc".
verbose 3.27 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.npmrc".
verbose 3.271 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.npmrc".
verbose 3.273 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\.npmrc".
verbose 3.278 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\.npmrc".
verbose 3.279 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\.npmrc".
verbose 3.28 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 3.281 Checking for configuration file "C:\\Users\\.npmrc".
verbose 3.283 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.yarnrc".
verbose 3.285 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 5.007 Error: https://registry.yarnpkg.com/typescript/-/typescript-3.3.3333.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat 'C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\npm-typescript-3.3.3333-171b2c5af66c59e9431199117a3bcadc66fdcfd6\\lib\\tsserver.js'"nd\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc
    at MessageError.ExtendableBuiltin (C:\Program Files (x86)\Yarn\lib\cli.js:243:66)
    at new MessageError (C:\Program Files (x86)\Yarn\lib\cli.js:272:123)pData\\Local\\Yarn\\Cache\\v2\\.tmp\\.yarnrc".
    at Extract.<anonymous> (C:\Program Files (x86)\Yarn\lib\cli.js:56849:14)a\\Local\\Yarn\\Cache\\v2\\.yarnrc".
    at Extract.emit (events.js:194:15)on file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.yarnrc".
    at Extract.module.exports.Extract.destroy (C:\Program Files (x86)\Yarn\lib\cli.js:131115:17)nrc".
    at onunlock (C:\Program Files (x86)\Yarn\lib\cli.js:130992:26)nd\\AppData\\Local\\.yarnrc".
    at C:\Program Files (x86)\Yarn\lib\cli.js:43373:25rs\\chtacklind\\AppData\\.yarnrc".
    at C:\Program Files (x86)\Yarn\lib\cli.js:43339:23rs\\chtacklind\\.yarnrc".
    at C:\Program Files (x86)\Yarn\lib\cli.js:56799:13acklind\\.yarnrc".
    at FSReqWrap.oncomplete (fs.js:153:21)ile "C:\\Users\\.yarnrc".
error https://registry.yarnpkg.com/typescript/-/typescript-3.3.3333.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat 'C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\npm-typescript-3.3.3333-171b2c5af66c59e9431199117a3bcadc66fdcfd6\\lib\\tsserver.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
PS C:\Users\chtacklind\Desktop\git\Project>

Sometimes with different errors:

...
[2/4] Fetching packages...
verbose 2.635 Performing "GET" request to "https://registry.yarnpkg.com/typescript/-/typescript-3.3.3333.tgz".
verbose 3.465 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.npmrc".
verbose 3.466 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 3.467 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc".
verbose 3.468 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.npmrc".
verbose 3.469 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\.npmrc".
verbose 3.47 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.npmrc".
verbose 3.471 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.npmrc".
verbose 3.473 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\.npmrc".
verbose 3.474 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\.npmrc".
verbose 3.48 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\.npmrc".
verbose 3.481 Checking for configuration file "C:\\Users\\chtacklind\\.npmrc".
verbose 3.482 Checking for configuration file "C:\\Users\\.npmrc".
verbose 3.483 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.yarnrc".
verbose 3.485 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 3.486 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 3.49 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\yarnrc".
verbose 3.492 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\6621674c4e43b664dde14df71eaf0cc8.09d44d8abc94f728f1c5ea93c22fe9b4f87d9076.prepare\\.yarnrc".
verbose 3.493 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.tmp\\.yarnrc".
verbose 3.494 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\.yarnrc".
verbose 3.495 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\.yarnrc".
verbose 3.496 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\.yarnrc".
verbose 3.497 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\Local\\.yarnrc".
verbose 3.501 Checking for configuration file "C:\\Users\\chtacklind\\AppData\\.yarnrc".
verbose 3.503 Checking for configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 3.504 Found configuration file "C:\\Users\\chtacklind\\.yarnrc".
verbose 3.505 Checking for configuration file "C:\\Users\\.yarnrc".
[1/4] Resolving packages...
[2/4] Fetching packages...
verbose 4.608 Error: EPERM: operation not permitted, unlink 'C:\Users\chtacklind\AppData\Local\Yarn\Cache\v2\npm-typescript-3.3.3333-171b2c5af66c59e9431199117a3bcadc66fdcfd6\.yarn-tarball.tgz'
error An unexpected error occurred: "EPERM: operation not permitted, unlink 'C:\\Users\\chtacklind\\AppData\\Local\\Yarn\\Cache\\v2\\npm-typescript-3.3.3333-171b2c5af66c59e9431199117a3bcadc66fdcfd6\\.yarn-tarball.tgz'".
info If you think this is a bug, please open a bug report with the information provided in "C:\\Users\\chtacklind\\Desktop\\git\\Project\\yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
PS C:\Users\chtacklind\Desktop\git\Project>

Notice that two yarn processes are running even though I have --mutex specified.

Of note, this package has a git dependency that has a tsc prepare step that needs to be performed. That package also has a git dependency that requires the same process. It's clear that yarn is trying to unpack the same package into to same place and we're getting a race condition.

Why is yarn running multiple instances even when told to not?

[MatthewGreenlaw]

As an update, using yarn install --network-concurrency 1 --mutex network seems to work every time where using one or the other options tends to succeed only part of the time.

[gitowiec]

So what is the resolution for this issue?
I use yarn 1.16 on Ubuntu Linux 18.04
And I still get this error message:

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/home/user/workspace/project/packages/components/node_modules/source-map-support'".

My command is:

yarn install --check-files --frozen-lockfile --network-concurrency 1

And I get this error once in two times :((
PS: I work in monorepo so I have enabled yarn workspaces
PPS:
I double checked
Adding --mutex file or --mutex network does not help.

[gitowiec]

Only solution I can confirm that works is putting in a script

until
    yarn install --check-files --frozen-lockfile;
do
    echo "Surprise, surprise. Let's try again..."
done

:(

[sarink]

Fwiw, I was able to switch to npm by doing nothing more than a find/replace of yarn with npm, using synp to convert yarn.lock to package-lock.json, and running npm install. I thought this would be a painful process, but npm has advanced a lot and it only took me about 30 min and now it Just Works everywhere

[cinderblock]

It seems like the problem here is that there isn't a truly simple example of what is going wrong. I posted a trivial package.json that reliably reproduces the problem for me but the packages included are somewhat complicated.

I suspect that the issue is when you have a dependency tree with shared "leaves"/tips that take time to compile/install (prepare) and yarn tries to do this prepare twice, concurrently. Each instance is "prepared" in a shared predictable location and they cannot both "write" to the same location at the same time (one deletes a file while the other expects it to still be there).

I've been meaning to create some simple proof of concept packages with inflated "prepare" steps to test this theory but haven't had the time. Maybe someone else can test this theory before I get to it?

[BYK]

This happens when you have multiple yarn instances running concurrently. You can use the --mutex option documented here: https://yarnpkg.com/en/docs/cli/#toc-concurrency-and-mutex

[cinderblock]

@BYK It seems there are many cases where this is not the problem. Others have even indicated that this option causes other problems for them.

[BYK]

@cinderblock #6650 seems like an edge case and should not really affect the resolution of this issue. The other example you mention is still yarn being triggered in install mode during another install, which probably is a problematic package as you should not trigger another installation during your package install. This can also be avoided by --ignore-scripts which is also recommended.

If you have any other leads, please do share so someone can hopefully spend more time to debug this.

[cinderblock]

@BYK I haven't seen anyone intentionally use yarn install inside of an install script. Did you see this trivial package.json that produces this problem? Granted it's something in the dependant packages that provokes this error, and maybe they have a buried install that you're referring to. However that package.json works just fine with npm...

Related, how/where is --ignore-scripts recommended? Many packages rely on post-install scripts to work.

[BYK]

Related, how/where is --ignore-scripts recommended? Many packages rely on post-install scripts to work.

Oh, I just recommended it here and I think many members of Yarn were vocal about this. 😀 Most packages are fine but there indeed are a few packages that rely on post-install scrpits yeah.

Did you see this trivial package.json

Yes but even a "trivial" package.json file can yield to a very large dependency tree so saying it is trivial, doesn't really change much except that I tend to interpret it in one of the following ways:

• yarn is too crappy so it cannot even deal with this _trivial_ package.json file
• this issue can be reproduced with a _trivial_ package.json file, yet you still cannot/do not fix it

where none of those are helpful so I tend to ignore that comment of yours.

For the actual issue, for yarn to run with the mutex, all yarn instances need to be called with the --mutex flag so the best way to see if this fixes the issue is adding --install.mutex network to your .yarnrc file (see https://yarnpkg.com/en/docs/yarnrc#toc-cli-arguments). That said this may yield to a deadlock if the initial install triggers another install, where the second install will wait for the main one to finish and the main one will be blocked on this script-invoked yarn to finish, hence I don't really know how to fix this issue other than implementing a thread/process-safe cache system, which is close to impossible with the locking primitives node provides. The closest thing seems like this proper-lock package but none of us had the time to try it out. I can help you if you are interested in trying to implement this into the cache write/read code.

[cinderblock]

@BYK Oh, my apologies if I came off like that. I had not seen a way of reliably reproducing the error before that, even if it is still not distilled enough to fix the problem. That was my entire impetus for that comment.

Forgive my persistence but I still don't see how the mutex option is a solution. I've tried running with mutex and it still ran yarn concurrently. Maybe I made a mistake in my testing? I expected running yarn --mutex ... would pass on that option to child instances (like make does, for instance). I have also tried your suggestion of adding --install.mutex network to my .yarnrc file to no avail (same errors). --verbose confirms the options are being loaded.

Maybe we could come at this from a different direction? What is yarn doing that npm is not? Why doesn't npm have this same issue?

[sarink]

@BYK, using the mutex flag is completely unacceptable, because there is an additional yarn bug which will then prevent you from _ever executing another yarn command_... This is like if your solution to "I locked my keys in my car" was, "don't worry, I thought of that! Just use this handy tool to bash in all your windows and door locks, now you'll never be able to lock your car again!" 🙄

Trivial package.json or not, this is a big problem with no solution or workaround that renders yarn completely unusable for a lot of people. It should get more attention. Especially considering it's been open for _two years_.

[BYK]

@sarink

@BYK, using the mutex flag is completely unacceptable, because there is an additional yarn bug which will then prevent you from ever executing another yarn command...

I am not aware of such a bug, can you point me to it if it is already reported? The way --mutex works is to prevent any other yarn instance using the same mutex from running, until the initial one finishes. So what you say (not your depiction) sounds like "works as expected" to me.

Trivial package.json or not, this is a big problem with no solution or workaround that renders yarn completely unusable for a lot of people. It should get more attention. Especially considering it's been open for two years.

Maybe you should take a minute to reflect on the internal contradiction of your own sentence: "renders yarn completely unusable for a lot of people" and "it's been open for two years". There are only 56 participants in this issue, which includes ~5 Yarn maintainers and a total of 138 comments, most of them circling around the same things. This is not "a lot of people" this is _some_ people and sure it is important to them but I see none of them takes this as important to send a single line of code fix and simply demanding fixes for a software that is provided for completely free to them.

@cinderblock

Forgive my persistence but I still don't see how the mutex option is a solution.

Nothing to forgive regarding persistence, it is actually to be celebrated as you are trying to solve the issue :)

I expected running yarn --mutex ... would pass on that option to child instances (like make does, for instance).

I am quite sure it is not passed down.

I have also tried your suggestion of adding --install.mutex network to my .yarnrc file to no avail (same errors). --verbose confirms the options are being loaded.

That is quite interesting. It maybe because the new yarn instance being triggered from another directory, ignoring your .yarnrc file. I may suggest using a global .yarnrc file with that option, that said I don't think that's a proper solution. We should only try this to see if it actually blocks the installation as we expected earlier.

Maybe we could come at this from a different direction? What is yarn doing that npm is not? Why doesn't npm have this same issue?

I appreciate the diverse thinking, that said Yarn and npm are so different in how they work, I don't think this really applies here. If we can identify which package triggers a yarn install as part of their own installation, we may figure out a solution. Maybe you can replace the yarn executable with some bash script which logs the invocation source, cwd and all arguments passed and then runs yarn as usual to get useful debug information and continue from there?

The ultimate solution would be a concurrency-friendly cache implementation as I mentioned earlier but with more debugging information, we may be able to find a cheaper workaround.

Thanks a lot for your cooperation with this @cinderblock, very much appreciated.

[Nopik]

The ultimate solution would be a concurrency-friendly cache implementation as I mentioned earlier but with more debugging information, we may be able to find a cheaper workaround.

I guess it should be possible to provide simplified concurrency handling - e.g. if other yarn instances are invoked during build step, there should be very little cache operations by top yarn process. At least, that is what I would expect. Making sure that cache gets flushed to disk before invoking any possible scripts might do the trick. Not sure, how complex is that to implement, though.

[cinderblock]

@BYK Oh! I had not considered the possibility that a sub-package was calling yarn ... in a script. Are you thinking that the reason npm install doesn't fail is because when its dependencies run yarn ... there is only one instance running?

[johannesschobel]

i managed to solve the issue with running

yarn cache clean
rm ./yarn.lock
yarn install

This process, however, takes ages, because it downloads all packages again because you a) don't have a cache any more and your lock file has been removed.

[entersb]

This might solve on your local machine, but problem with yarn remains in case it is used on bitrise. It is clean image, from scratch. Network concurrency is required in any case, even when single yarn process is running.

[TikiTDO]

@BYK

Tere are only 56 participants in this issue, which includes ~5 Yarn maintainers and a total of 138 comments, most of them circling around the same things.

That seems like a pretty high number for this repo. It's the highest comment total among all open and closed issues, and it's among the highest participant counts.

What more, while researching a (probably) related issue I saw quite a few people with issues that were likely related. The unfortunate thing is that a lot of these people either gave up on yarn as a whole, or swallowed the cost of doing --network-cocurrency 1 whenever the yarn lock file changed. That's exactly what I was doing for one of my projects until I finally figured out that it was a sub-script.

This is not "a lot of people" this is some people and sure it is important to them but I see none of them takes this as important to send a single line of code fix and simply demanding fixes for a software that is provided for completely free to them.

Yarn isn't the type of project that people can easily jump into. It's a complex system that does a whole lot of things in asyncronous ways, meaning you can't just plop a debugger and walk up the stack to see what's going on. As such, it is by no means easy to understand this code base, and it's even harder to modify it. Hell, at this point I've already spent a few days of cumulative time reading through the code, and I still don't feel confident enough to submit even a simple a patch with relevant tests. Given that I'm sitting on over two decades of experience, and that code reading is one of my specialties, I wouldn't give an average dev the greatest chances.

In other words, what may be a quick, simple one-line fix to you can be a big multi-day project for someone that's not familiar with the inner-workings of the project. This is before I mention the implicit policies that exist in various communities. I've had at least a few PRs denied in various open source projects because I didn't follow some protocol, or write the right test, obey the right spec, or even for just having a fix not up to some undocumented coding standards. It's can be a challenge contributing meaningfully to large projects as an outsider.

If this really is a single-line fix for you, wouldn't it be quicker to just write that fix instead of writing a long post criticizing others for not doing it?

If we can identify which package triggers a yarn install as part of their own installation, we may figure out a solution.

I have an example of a package that exibits this sort of behavior here, though I don't see a yarn install in there (unless the bob build does it, though it also showed this problem back when the command was "prepare": "node ./scripts/generate-mappings",).

The ultimate solution would be a concurrency-friendly cache implementation as I mentioned earlier but with more debugging information, we may be able to find a cheaper workaround.

A great starting point would be a warning if such a situation is detected (assuming it is detectable). A concurrency friendly cache seems like it would have to be a concentrated effort by at least one member of the yarn community.

[dgobaud]

Update found a workaround that does work.... a git submodule and then have the location of the package be the local folder. Not ideal but it works.

We're having this problem too in CircleCI and it is a big problem. The problem seems to be related to using a package hosted on github.com and perhaps Linux (it works on OSX). mutex and network-concurrency options do nothing.

"my-js-lib": "ssh://[email protected]:dgobaud/my-js-ib#1.0.0"

if I remove that it works in CircleCI. Locally it works with it on OSX with yarn 1.17.0.

But it will not work on CircleCI with node 12.8.1 and yarn 1.17.3 (circle image circleci/node:latest)

Or with Node 8.15.0 and yarn 1.12.3 (circle image circleci/node:8.15.0)

#!/bin/bash -eo pipefail
yarn install --mutex network --network-concurrency 1
yarn install v1.12.3
[1/4] Resolving packages...
warning Resolution field "[email protected]" is incompatible with requested version "mixin-deep@^1.2.0"
warning Resolution field "[email protected]" is incompatible with requested version "set-value@^2.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "set-value@^0.4.3"
[2/4] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
$ cd functions && yarn install
yarn install v1.12.3
[1/5] Validating package.json...
[2/5] Resolving packages...
warning Resolution field "[email protected]" is incompatible with requested version "mixin-deep@^1.2.0"
warning Resolution field "[email protected]" is incompatible with requested version "set-value@^2.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "set-value@^2.0.1"
[3/5] Fetching packages...
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.15.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/home/circleci/.cache/yarn/v4/npm-lodash-4.17.15-b447f6670a0455bbfeedd11392eff330ea097548/node_modules/lodash/_arrayReduceRight.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
info There appears to be trouble with your network connection. Retrying...
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Exited with code 1

[revmischa]

--network-concurrency 1 succeeds, --network-concurrency 8 fails, on circleCI/node:10

[FezVrasta]

May someone help me understand why Yarn needs to fail on EEXIST and EOENT?

In case of EEXIST I would expect Yarn to warn about it, and then override the file.
In case of EOENT, I would expect Yarn to create the missing folder (which is usually the cause of the issue).

I understand it may have side effects, so maybe this behavior could be made stricter with a flag (or vice-versa).

But what's the point of keeping these errors around? They are no useful to anyone.

[sarink]

@BYK Sorry, I just saw your comment

I am not aware of such a bug, can you point me to it if it is already reported? The way --mutex works is to prevent any other yarn instance using the same mutex from running, until the initial one finishes. So what you say (not your depiction) sounds like "works as expected" to me.

This is the bug: https://github.com/yarnpkg/yarn/issues/6650 as mentioned in https://github.com/yarnpkg/yarn/issues/2629#issuecomment-481297806 (which I believe is now buried under the "show more history" in this thread)

There are only 56 participants in this issue

Well, I guess, that's a fair point

[gitowiec]

This bug is still in yarn v1.19.1. I don't get it why Yarn Team does not repair this very bothersome bug.
here is my .yarnrc and it does not help:

save-prefix ""
--install.check-files true
--add.check-files true
--remove.check-files true
--install.frozen-lockfile true
--add.frozen-lockfile true
--remove.frozen-lockfile true
--install.mutex network
--install.mutex file

What I just discovered that running npx lerna clean && ./yarn-install-in-loop.sh helps.
Cleaning (removing) all node_modules directories in my monorepo does help.

[realtime-neil]

@gitowiec can confirm. My containerized yarn install invocations are data racing something on the file system, and every attempt to limit yarn to a single .yarnrc file fails. I'm giving up and going back to npm.

[pesimeao]

I've found that my problem was having package dependencies from git repositories through multiple levels (ie. my package -> git package -> git package -> git package). Also, the cache during installation didn't work well comparing to npm (npm only checkout once, but yarn checkout multiple times the same package during the same installation).
I'm moving back to npm. It works well since v6.

[skay97]

A mentioned above, this issue still exists. Here is what we have added to our config.yml for circleci to make our tests pass:
- run: name: Yarn Install source ~/setyarnpath.sh i=5; until yarn; do echo "Yarn failed. Retrying..."; ((i--)); if [[ "$i" == '0' ]]; then break; fi; done

[evsasse]

I was having the same problem. Using macOS, and docker-compose, with a host[1] volume that had my code AND node_modules inside it.

Changed node_modules to be inside an anonymous volume, but I guess a named volume would also work. And it seems to be working fine now, and also installing the dependencies MUCH faster.

My docker-compose file changed from:

services:
  ...
  web:
    build: .
    volumes:
      - .:/home/example
    ports:
      - "3000:3000"
    ...

To:

services:
  ...
  web:
    build: .
    volumes:
      - .:/home/example
      - /home/example/node_modules
    ports:
      - "3000:3000"
    ...

[1] https://success.docker.com/article/different-types-of-volumes

[cinderblock]

I'm seeing a new error with recent yarn that I believe is a new symptom of this problem.

yarn stdout [1/4] Resolving packages...
yarn stdout [2/4] Fetching packages...
yarn stderr error https://registry.yarnpkg.com/prettier/-/prettier-1.19.1.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EEXIST: file already exists, mkdir '/home/pi/.cache/yarn/v6/npm-prettier-1.19.1-f7d7f5ff8a9cd872a7be4ca142095956a60797cb-integrity/node_modules/prettier'"
yarn stdout info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
yarn stderr Process stalled
yarn stderr Active handles:
yarn stderr   - Socket
yarn stderr   - Socket
yarn stderr   - Socket
yarn stderr   - TLSSocket
yarn stderr   - TLSSocket
yarn stderr   - TLSSocket

Note: yarn stderr/out are the prefix that my program gives to yarn's output in my env

I believe this is the same problem because my project could pretty consistently create the old symptoms in the same way that these new symptoms are happening (and the old symptoms have stopped).

For reference, this happens on install after I clear the yarn cache and node_modules or update one particular git package dependency.

The particular package dependency is a dependency of another git package that I actually depend on (both have similar prepare steps that rely on TypeScript). If I make changes to these dependencies on #master and do a yarn upgrade --latest, the problem occurs (and on subsequent yarn install.

When I update that sub package manually (into a completely separate node_modules folder!), the yarn install works again. This makes me think that yarn is accidentally using the cache by two processes concurrently and this is causing the problems we're seeing in this issue.

[pesimeao]

This happens to me when using 2 or more packages installed by git dependency. Somehow there are multiple processes to the same package running the prepare script. Also, it starts failing with npm also in the last releases.

Dependencies:
A -> B & C (both by git, with prepare script)
B -> C (by git, with prepare script)

[gregory]

This has bene opened for years & still happens with yarn 1.22.0.
I've just spend hours trying to debug what was going on without any luck, and it seems I haven't been the only one.

The only solution I'm seeing now is to switch to npm.

[mathieucaroff]

@gregory In my case, I remember in June 2019, yarn would always end up installing the needed packages, even if it took 2-4 re-runs to get there. Furthermore, even with these re-runs, yarn was still faster than npm.

I'd re-run until yarn finishes using commands like this:

while ! yarn install; do echo --- ; done

[dgobaud]

The easy fix for us was publishing a private package and using it instead of a git link. Still annoying though

[gregory]

while ! yarn install; do echo --- ; done

Really sad that the only fix is brute force ... unbelievable that no one fixed this yet.

cc @arcanis

[gregory]

Trying two subsequent yarn install, works the first time, then fails.

Fast, reliable, and secure dependency management.

That's unreliable.
```[3/5] Fetching packages...
error https://registry.yarnpkg.com/lz4/-/lz4-0.6.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, link '/app/.cache/yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/build/Release/lz4.node' -> '/app/.cache/yarn/v6/npm-lz4-0.6.3-78
df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/lz4.node'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info There appears to be trouble with your network connection. Retrying...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
[5/5] Building fresh packages...
$ npm run prepare:mjs && npm run prepare:js

[email protected] prepare:mjs /app/.cache/yarn/v6/.tmp/43563e016bb56318ebd76037a0f6ce2f.73d5f4dbffab6f6a27f26c6611e32662c98c2891.prepare
BABEL_ESM=1 babel src -d . --keep-file-extension

Successfully compiled 39 files with Babel.

[email protected] prepare:js /app/.cache/yarn/v6/.tmp/43563e016bb56318ebd76037a0f6ce2f.73d5f4dbffab6f6a27f26c6611e32662c98c2891.prepare
babel src -d .

Successfully compiled 39 files with Babel.

[3/5] Fetching packages...
error https://registry.yarnpkg.com/lz4/-/lz4-0.6.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, link '/app/.cache/yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/build/Release/lz4.node' -> '/app/.cache/yarn/v6/npm-lz4-0.6.3-78
df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/lz4.node'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
info There appears to be trouble with your network connection. Retrying...

```

[gregory]

Anyone has any idea why yarn would call

error https://registry.yarnpkg.com/lz4/-/lz4-0.6.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, link '/app/.cache/yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/build/Release/lz4.node' -> '/app/.cache/yarn/v6/npm-lz4-0.6.3-
  df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/lz4.node'"

when the right path should have been:

/app/.cache/yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/lz4.node

it seems yarn is adding obj.target/build/Release/ for some reason. Could be related to https://github.com/yarnpkg/yarn/commit/0e7133ca28618513503b4e1d9063f1c18ea318e5

[tombh]

I was getting this same frustrating and hard to debug error. The problem in my case seemed to be yarn workspace behaviour caused by different versions of the same dependency in different packages (specifically ava versions 2 and 3). Only once I'd upgraded all occurrences of ava to their latest did I stop getting this error.

[Gervwyk]

I'm running 1.22.4 and was stuck on this issue for hours. Our monorepo has several modules using the same packages. Finally got it sorted by applying the following:
1) Make sure you use the same version of a package accross all modules - this will cause a crash for sure, even on devDependencies.
2) Pin all versions in the monorepo in all package.json files.

[k3n]

Can confirm still a problem on 1.22.4. Originally, it was mocha, and after making sure all packages were using the same version, I'm now getting errors generated from camelcase which I don't even use in my project -- apparently it's from yargs, possibly from Lerna.

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/code/project/src/packages/private-package/node_modules/camelcase'".

[bnbarak]

May I ask if there is a solution in sight? We keep deleting 20 node_modules and a yarn.lock to fix it.

[clero]

May I ask if there is a solution in sight? We keep deleting 20 node_modules and a yarn.lock to fix it.

I personally have switched to lerna regarding workspace handling.

[k3n]

Fairly sure that current Lerna simply passes-thru to Yarn.

I was able to work-around my issues by adding a nohoist config for Ember packages -- my current env is using an older version of Ember, which is incompatible with workspaces.

    "nohoist": [
      "**/ember-package/*ember*",
      "**/ember-package/*ember*/**",
      "**/ember-package/loader.js"
    ]

[cinderblock]

I think we've got a minimal repro here now: https://github.com/yarnpkg/yarn/issues/7212#issuecomment-637978197

[costinel-draculet]

Deleting yarn.lock then yarn install worked for me as well

[cadavre]

Any news here? Our CI process just went down with all pipelines failing because of dependency installation failure from yarn. That's ridiculous.

Setting --network-concurrency fixes nothing and jobs run on clean machines (no node_modules, no yarn .cache).

[merceyz]

@cadavre Not making any guarantees but this might not be a problem in v2, you can try it with

yarn set version 2 && yarn config set nodeLinker node-modules

https://yarnpkg.com/getting-started/install#per-project-install
https://yarnpkg.com/configuration/yarnrc#nodeLinker

[garyo]

This just started happening to me too, after upgrading some of my vue & firebase dependencies. Now 100% repeatable in my CI and dev machines. Adding --network-concurrency 1 does not reliably fix it. I'm not out of disk space or inodes. I'm on WSL1. Yarn 1.22.4.

[cadavre]

I fixed it by temporarily changing cache directory, that I delete just afterwards.

For me it's Docker build:

RUN yarn install --check-files --cache-folder .ycache && rm -rf .ycache