yarn 1.9.2 takes a long time on linking dependencies

[kenips]

Hello yarn team,

Just upgraded to 1.9.2 and running yarn add next-images in my app it took a long time at "Linking dependencies...". Reverting back to 1.7.0 is fine.

yarn 1.9.2: Done in 375.98s
yarn 1.7.0: Done in 36.15s

Attached my package.json here for reference.

Steps to reproduce:

1. mkdir -p test
2. Copy in package.json
3. Run yarn
4. Run yarn add next-images

Environment:
macOS 10.13.6

[domiSchenk]

same with version 1.9.1/1.9.2 on windows 10

[arcanis]

I'll take a look today

[alvr]

Yeah, I got this problem too on Windows 10. I just tried with this:

yarn add -D superagent

Yarn 1.7.0:

info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 4.11s.

Yarn 1.9.2:

info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 283.12s.

[arcanis]

Did it happen to anyone on Linux or osx, by any chance?

On Mon, Jul 30, 2018, 9:37 AM Álvaro S. [e-mail] wrote:

Yeah, I got this problem too on Windows 10. I just tried with this:

yarn add -D superagent

Yarn 1.7.0:

info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 4.11s.

Yarn 1.9.2:

info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 283.12s.

—
You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/6177#issuecomment-408789307, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA_Wa5mWxo8kqQudhg4EMHK2sX45WmOAks5uLsW4gaJpZM4Vlhn3
.

[alvr]

On the same computer and command above but running Ubuntu 18.04 on WSL:

info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 8.96s.

[arcanis]

So Windows adds 275s? o_O I have an idea what might cause this (https://github.com/yarnpkg/yarn/pull/6010), I'll test this as soon as I get on my Windows laptop.

[arcanis]

Hm, I just tried on OSX and didn't reproduce this issue 😞 0.90s with both superagent and next-images

[domiSchenk]

just testet with a new angular project:

ng new test-app --skip-git --skip-install
npx [email protected] install
─> 63.77s (as expected ok for first time)
npx [email protected] add superagent
─> hangs at linking 0/1118
Done in 379.69s.

This on windows 10

[arcanis]

While I don't have the same variation as you (I respectively get 4s vs 10s), I do get a significant delta increase. I'm almost positive that this is due to #6010, and some debug seem to confirm this. Seems like the check was heavier than I anticipated.

The fix will involve setting the timestamp to something sane at unpack time. What's annoying is that it will require to bump the cache version, since the old one has "corrupted" dates ...

[kenips]

@arcanis my original reproduction was on macOS 10.13.6. Let me see if I can test without #6010.

[kenips]

@arcanis I can confirm that reverting #6010 on 1.9.2 fixes the issue on macOS.

[SEAPUNK]

Likewise; #6010 completely wrecked our install times, esp. considering we install a lot of modules across a few workspaces. A few of my coworkers also report that node completely hangs for a good while before it's able to continue installation.

[arcanis]

Thanks everyone. I'll make a priority of fixing this issue, and hopefully a fix will land in 1.9.3, by friday at the latest (we can't just revert it, since the underlying problem is arguably worse, causing wrong packages to be installed).

[arcanis]

I've opened #6204 that should fix the issue. Can someone here please confirm that it clears the regression you witnessed? I don't see it as much as you, so I'd like to be sure it also fixes those horrible install time you have.

[rfuhrer]

@arcanis How would I install that branch to test? I tried npm i -g https://github.com/arcanis/yarn.git#fix-perfs but I get Error: Cannot find module '../lib/cli' whenever I try to run yarn.

[arcanis]

You can use the prebuilt yarn-1.10.0-0.js file from the following zip:

artifacts.zip

Something like node ~/yarn-1.10.0-0.js install should do the trick.

[rfuhrer]

1. I tried to install a small library on 1.9.2 and it took 405 seconds.
2. used your branch to remove it which triggered a whole cache redo, which was slow but you mention that in your pull request that we can't trust the cache timestamps
3. used the branch again to add that same library: 17 seconds.

Looks like this approach is the way to go on Windows 10 anyways.

Edit: noticed my yarn.lock now has lines like:

integrity sha512-p7e4wE+a1AxlfCJQL1IIBltblV9VqFSMlUuPW3PUp0fguo0yaTv9paY5WlFwrj0YhypBj3zHcjSdIruHrgbErg==

If we're going away from full file checks again, isn't this not necessary? Or is this from another 1.10 change that's unrelated?

[arcanis]

Yep, the integrity line is a new change that'll get added in a later release (not entirely sure whether it'll be the 1.10 or 2.0). It won't be part of the 1.9.3, which will simply cherry-pick #6204.

[kenips]

@arcanis confirm that this fixes the issue for me

[stevethatcodes]

Can confirm this is also impacting OSX with Yarn 1.9.2

yarn add redux-devtools -D --verbose completing in between 88 - 120 seconds.

[peroperje]

Debian 8
node v8.11.3
yarn 1.9.2
[3/4] Linking dependencies - sometimes take a lot of time, sometimes even completely frozen.

[acao]

Still having this issue with:

• OSX High Sierra 10.13.6
• Node 8.11.3, 9.11.2
• Yarn 1.10-x nightly (in my case, 1.10.0-20180802.0946)

[SEAPUNK]

@acao I'm pretty sure that nightlies are built off latest master, and the fix is still in a PR (hasn't been merged yet). Try the explicitly built version: https://github.com/yarnpkg/yarn/issues/6177#issuecomment-409715910

[acao]

@SEAPUNK ah yes #6177 (comment) does work, thanks! Here are some explicit instructions for this temporary workaround for folks who are less shell-savvy, and not using windows.

1. Download or curl https://github.com/yarnpkg/yarn/files/2251068/artifacts.zip to ~/
   
   
   
   • (aka /Users/myuser/ or /user/myuser/ for osx/linux)
   
   
2. unzip to ~/artifacts
3. mv ~/artifacts/yarn-1.10.0-0.js ~/yarn/bin/yarn
4. rm -rf ~/artifacts ~/artifacts.zip
5. source ~/.bashrc, or .zshrc or reset your shell session however.

Now, yarn install and other commands should work fine, without the performance regression. yarn -v will resolve to 1.10.0-0

[arcanis]

Fixed in the 1.9.4, now available! Let me know if that fixes the issue for you 🙂

[weishijun14]

I can't update to 1.9.4!!!

Why I did this:

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

and I update like this :

sudo apt-get update

result:

[rictorres]

tried with v1.9.4 and also with v1.10.0 and still having the same issues.

yarn install v1.10.0-0
success Saved lockfile.
✨  Done in 318.50s.

npm install
added 1588 packages in 62.024s

[jasonadkison]

Linking dependencies is still super slow for me using 1.9.4.

[kthakore]

Yeah it is still having a huge problem on v1.9.4

[chriswatrous]

If is anyone still having problems, check the yarn version that's actually being run with yarn -v.

On my system (macOS) I had [email protected] installed at ~/.yarn/bin/yarn and that was shadowing the [email protected] installed at ~/.nvm/versions/node/v8.11.4/bin/yarn.

Use which -a yarn to see if you have more than one installed.

[shyaaam]

Still having a problem with 1.10.0 . Very frustrating, and my boilerplate is tightly tied to the yarn system. Im stuck with this stupid thing. npm was far better.

[arcanis]

Then use it, or help us understand what causes the slowdown you're experiencing. Noone will fix a problem without clear reproduction steps.

Also please open a new issue if you wish to help us. The issue described in this thread has been solved quite some time ago.

[arcanis]

Locking this issue. If someone reads this and still get slowness with the 1.10 and above and is interested into mutual collaboration to fix it (that includes either a reproduction or an explanation as to where the problem comes from on your machine), please open a new issue.