Yarn: `yarn` fails with EMFILE on large projects (CentOS 5.11)

[mattdsteele]

Do you want to request a _feature_ or report a _bug_?
Bug

What is the current behavior?
Running yarn on a reasonably large project fails with Error: EMFILE: too many open files, uv_resident_set_memory

$ yarn
yarn install v0.15.1
[1/4] Resolving packages...
[2/4] Fetching packages...
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ 0/1096/privdir/duit999/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/reporters/base-reporter.js:95
    var _process$memoryUsage = process.memoryUsage();
                                       ^

Error: EMFILE: too many open files, uv_resident_set_memory
    at Error (native)
    at ConsoleReporter.checkPeakMemory (/privdir/duit999/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/reporters/base-reporter.js:95:40)
    at Timeout.peakMemoryInterval.setInterval [as _repeat] (/privdir/duit999/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/reporters/base-reporter.js:90:12)
    at Timeout.wrapper [as _onTimeout] (timers.js:425:11)
    at tryOnTimeout (timers.js:232:11)
    at Timer.listOnTimeout (timers.js:202:5)

By "reasonably large" my project's yarn.lock is 280K. The project installs with npm fine on CentOS, and with yarn fine on my Windows workstation.

If the current behavior is a bug, please provide the steps to reproduce.

1. Run yarn on a project with a reasonably large dependency set.
2. It's hard to tell exactly where the limit is. yarn init works fine in an empty folder, but it failed on a project with a 175K yarn.lock file. It also fails on that same project after I delete the yarn.lock file and attempt to rebuild.

Our yarn.lock contains private & proprietary packages so I can't paste it here, but I assume it will fail in this environment with any reasonably large dependency set.

What is the expected behavior?

Installing dependencies via yarn should work as it does on Windows.

Please mention your node.js, yarn and operating system version.

• yarn 0.15.1
• CentOS 5.11, 8GB ram
• Node 6.7.0, npm 3.10.3 (installed via nvm)
• Also fails with Node 5.9.1, npm 3.7.3

[eanakashima]

I'm running into the same issue on OS X. I tried raising ulimit -n but it seems that there is no way to raise it past 1048 on my combination of device and os, and that doesn't seem to be high enough. Our yarn.lock is about 250kb. If I remove dependencies until it's under about 150kb, then I can get it working again.

Please mention your node.js, yarn and operating system version

• yarn 0.15.1
• OS X 10.12 (MacBook Air)
• Node 6.8.1, npm 3.10.8 (installed via nvm)

[jgerigmeyer]

I'm also hitting this issue on OS X, with a ulimit -n of 1024. Our yarn.lock file is 249kb. This is basically a blocker for using yarn for large projects.

Please mention your node.js, yarn and operating system version.

• yarn 0.16.1
• OS X 10.12.1
• Node 7.0.0, npm 3.10.8

[eanakashima]

I was able fix this by removing a line from my bash profile that set ulimit -n.

[cvbuelow]

@eanakashima that fixed it for me too, thanks

[bestander]

Also you can use --network-concurrency flag to limit the amount of parallel downloads