Yarn: Linking dependencies very slow on v1.9.4

[JaKXz]

Continued from https://github.com/yarnpkg/yarn/issues/6177#issuecomment-414337411

I am seeing very slow linking steps in both fresh installs as well as using yarn when the lockfile was updated and pulled from github.

Sorry if I'm duplicating an issue but I didn't see this report for v1.9.4 which is supposed to have a fix?

on macOS 10.13.6 with a 15" mid-2015 MBP:

yarn install v1.9.4
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning " > @storybook/[email protected]" has unmet peer dependency "babel-runtime@>=6.0.0".
warning " > [email protected]" has incorrect peer dependency "eslint-plugin-react@^6.6.0".
warning " > [email protected]" has incorrect peer dependency "react-dom@^15.4.2".
warning "react-hot-loader > [email protected]" has incorrect peer dependency "react@^0.14.0 || ^15.0.0".
warning "react-hot-loader > [email protected]" has incorrect peer dependency "react-dom@^0.14.0 || ^15.0.0".
warning " > [email protected]" has incorrect peer dependency "react@^0.14.0 || ^15.0.0".
warning " > [email protected]" has incorrect peer dependency "react-dom@^0.14.0 || ^15.0.0".
warning " > [email protected]" has unmet peer dependency "react-router@^4.0.0".
[4/4] 📃  Building fresh packages...
✨  Done in 506.17s.

[arcanis]

Unfortunately I haven't been able to witness this slowdown by myself - for some reason my system seems much faster than yours. My fix has landed in the 1.9.4, and I got reports confirming that it worked (plus there has been a lot less traffic on this issue than before), so I'm not sure what might cause this for you. Can you describe the symptoms a bit more? For example:

• Does the progress bar still hangs before starting?
• Do you reproduce this effect on the 1.8?
• Is it slow when you have a node_modules?
• Is it slow when you remove node_modules/.yarn-integrity?

[JaKXz]

1. The progress bar does not hang before starting.
2. Yes, I was able to reproduce on older versions.

3 & 4. The time did improve significantly to 82.89s after:

rm -rf node_modules
yarn

I should have added that this is in a repo with 111 dependencies, so a guess is that the old node_modules/.yarn-integrity was the issue.

devDeps + dependencies

  "devDependencies": {
    "@storybook/addon-actions": "^3.4.1",
    "@storybook/addon-knobs": "^3.4.8",
    "@storybook/addon-storyshots": "^3.4.1",
    "@storybook/addons": "^3.4.1",
    "@storybook/react": "^3.4.1",
    "autoprefixer": "6.5.1",
    "axios-mock-adapter": "1.7.1",
    "babel-core": "^6.26.3",
    "babel-eslint": "^8.2.5",
    "babel-jest": "^23.0.1",
    "babel-loader": "^7.1.4",
    "babel-plugin-syntax-dynamic-import": "^6.18.0",
    "babel-plugin-transform-object-rest-spread": "^6.26.0",
    "babel-preset-env": "^1.7.0",
    "babel-preset-react": "^6.24.1",
    "babel-preset-react-optimize": "^1.0.1",
    "bootstrap-loader": "^2.2.0",
    "cheerio": "^0.22.0",
    "classnames": "^2.2.5",
    "create-react-class": "^15.6.0",
    "css-loader": "^0.26.1",
    "enzyme": "^3.3.0",
    "enzyme-adapter-react-16": "^1.1.1",
    "enzyme-to-json": "^3.3.1",
    "eslint": "^3.12.2",
    "eslint-config-airbnb": "^13.0.0",
    "eslint-import-resolver-webpack": "^0.8.1",
    "eslint-loader": "^1.6.1",
    "eslint-plugin-import": "^2.2.0",
    "eslint-plugin-jsx-a11y": "^2.2.0",
    "eslint-plugin-react": "^7.1.0",
    "extract-text-webpack-plugin": "^3.0.0",
    "file-loader": "^0.10.0",
    "html-webpack-plugin": "^2.28.0",
    "identity-obj-proxy": "^3.0.0",
    "jest": "^22.4.3",
    "jest-enzyme": "^6.0.1",
    "jsdom": "^9.12.0",
    "moment-timezone": "^0.5.14",
    "node-sass": "^4.5.2",
    "npm-run-all": "^4.0.2",
    "nsp": "^3.2.1",
    "postcss-loader": "^1.3.1",
    "react-addons-test-utils": "^15.6.0",
    "react-hot-loader": "^3.1.3",
    "react-loader": "^2.4.2",
    "react-test-renderer": "^16.2.0",
    "redux-devtools-extension": "^2.13.2",
    "redux-fsa-linter": "1.0.2",
    "redux-immutable-state-invariant": "1.2.3",
    "redux-mock-store": "1.2.1",
    "resolve-url-loader": "^1.6.0",
    "sass-lint": "^1.10.2",
    "sass-loader": "^6.0.2",
    "storybook-react-router": "^1.0.1",
    "style-loader": "^0.18.2",
    "stylelint": "^8.4.0",
    "stylelint-config-css-modules": "^1.1.0",
    "stylelint-config-standard": "^18.0.0",
    "svg-react-loader": "^0.4.5",
    "url-loader": "^0.5.7",
    "webpack": "^3.5.4",
    "webpack-bundle-analyzer": "^2.9.0",
    "webpack-dev-server": "^2.7.1",
    "webpack-manifest-plugin": "^1.3.1",
    "webpack-merge": "^3.0.0",
    "webpack-node-externals": "^1.5.4",
    "why-did-you-update": "^0.1.1"
  },
  "dependencies": {
    "@fortawesome/fontawesome-svg-core": "^1.2.0-14",
    "@fortawesome/free-regular-svg-icons": "^5.1.0-11",
    "@fortawesome/react-fontawesome": "^0.1.0-11",
    "@jakxz/redux-data-structures": "^0.2.0",
    "airbnb-prop-types": "^2.8.1",
    "airbrake-js": "0.5.9",
    "axios": "0.15.2",
    "babel-polyfill": "^6.26.0",
    "bootstrap-sass": "^3.3.7",
    "history": "^4.7.2",
    "humps": "^2.0.1",
    "immutability-helper": "2.0.0",
    "json-api-normalizer": "^0.4.1",
    "lodash": "^4.13.1",
    "map-props-to-children": "^1.0.0",
    "moment": "^2.18.1",
    "prop-types": "^15.5.10",
    "pui-react-tooltip": "^8.3.1",
    "query-string": "^5.1.0",
    "react": "^16.2.0",
    "react-addons-shallow-compare": "^15.6.0",
    "react-composer": "^5.0.0",
    "react-dates": "^12.1.2",
    "react-dom": "^16.2.0",
    "react-helmet": "^5.2.0",
    "react-loadable": "^5.4.0",
    "react-moment-proptypes": "^1.4.0",
    "react-redux": "^5.0.5",
    "react-router-dom": "^4.2.2",
    "react-router-prop-types": "^0.0.3",
    "react-router-redux": "^5.0.0-alpha.9",
    "react-select": "^1.2.1",
    "react-table": "^6.8.6",
    "recompose": "^0.26.0",
    "redux": "3.5.2",
    "reselect": "^3.0.1",
    "segseg": "^0.2.2",
    "victory-chart": "^24.6.1",
    "victory-core": "^20.6.0"
  }

I'd be happy to mark this issue as resolved, if you are satisfied @arcanis. Thanks for your help!

[arcanis]

Yes, I was able to reproduce on older versions.

Ok, so it's a different issue from the other one. That's super weird. The integrity file make things faster, not slower, it doesn't make much sense 😞

The only thing I could see was if you had used --check-files at some point, which I think cause Yarn to double-check that the files in your node_modules are all there ... but it's a somewhat obscure option so I doubt it's the case.

[ItsNoHax]

yarn cache clean

Fixed it for me. Installing dependencies went from: 200s -> 32s

Nevermind, the next install took a long ass time too. Some weird behaviour going on.

[tonkotsuboy]

Is there progress something?

[adamscybot]

Just adding my experience that on Windows 10 the update did in fact cause a massive speedup.

[cvharris]

I have a brand new Mac and installed yarn 1.9.4 and every fresh yarn with no yarn.lock or node_modules results in a long install time, over 2 minutes

[adamscybot]

Actually yeh. On OSX, im seeing very slow linking. New Mac. Sometimes 3-4 minutes. Generally, NPM does it in half of time for same deps.

[arcanis]

Yarn 1.10 uses copy-on-write when possible, which should be a bit faster on OSX

[shyaaam]

Today, it took 50minutes to just upgrade a package in my reactjs project. Not the first time. Thinking of ditching yarn. v 1.9.4

[arcanis]

Yarn is a community project, and as such the best would be for someone experiencing the slowdown to investigate. There's not much I can do on my side since I don't reproduce this problem.

Also note that the 1.9.4 required us to bump the cache version. Because of this, any install needs to generate a new cache, hence an expected slowdown for the first install you run on your projects (in which case most of the time will be spent in the "fetching" step). Finally, I'd recommend you to try out the 1.10 if you have issues, since the copy on write might solve the problem altogether.

[JaKXz]

I think it's best to close this issue since v1.10 is what's considered stable. Thanks again for your help @arcanis!

[shyaaam]

I was using this repo https://github.com/kriasoft/react-starter-kit which depends on yarn. Initially i installed v1.9.2, which was months ago. When i googled the solution, i saw updating to 1.9.4 would make it go away. Surprise, It didn't. Then again i saw comments that this issue is fixed in v1.10. It still wasn't. For taking 45 minutes just to add a package now and then, it consumed too much time for my project schedules. Apparently you guys now know why users here are venting their frustration. I have given up hope on this being fixed.
If you can, pls advise me steps on how to ditch yarn and revert to npm for a project thats using yarn. THAT would certainly be a big help, thanks. Im a newbie.

[JaKXz]

@arcanis it seems that this issue is attracting a lot of toxicity, it's probably best to lock it at this point. 😞 sorry about that.

@pihyper run rm -rf node_modules yarn.lock && npm install. I also suggest to everyone using a little bit more respect in your comments, especially when asking for help [not to mention for free].

[arcanis]

No worry @JaKXz, it happens from time to time 🤷‍♀️

Locking this issue. If someone reads this and still get slowness with the 1.10 and above and is interested into mutual collaboration to fix it (that includes either a reproduction or an explanation as to where the problem comes from on your machine), please open a new issue.