Build-image: Yarn install runs even when a node_modules cache is found

[jlengstorf]

Thanks for reporting this bug!

I could have sworn we already had an issue open for this, but I can't find it. Filing here so I can link when people hit this.

Describe the bug

The build will restore node_modules successfully, but when a yarn.lock file is present it re-runs the yarn command and reinstalls all packages anyways.

Here's an excerpt of a log from this community issue that shows the unnecessary install happening and adding almost 4 minutes to the build:

8:06:33 PM: Started restoring cached node modules
8:06:33 PM: Finished restoring cached node modules
8:06:33 PM: Started restoring cached yarn cache
8:06:33 PM: Finished restoring cached yarn cache
8:06:34 PM: Installing NPM modules using Yarn version 1.17.0
8:06:35 PM: yarn install v1.17.0
8:06:35 PM: [1/4] Resolving packages...
8:06:37 PM: [2/4] Fetching packages...
8:07:59 PM: info [email protected]: The platform "linux" is incompatible with this module.
8:07:59 PM: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
8:07:59 PM: info [email protected]: The platform "linux" is incompatible with this module.
8:07:59 PM: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
8:07:59 PM: info [email protected]: The platform "linux" is incompatible with this module.
8:07:59 PM: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
8:07:59 PM: [3/4] Linking dependencies...
8:07:59 PM: warning " > @crello/[email protected]" has incorrect peer dependency "react@~16.9.0".
8:07:59 PM: warning " > @crello/[email protected]" has incorrect peer dependency "react-dom@~16.9.0".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "webpack@>=2".
8:07:59 PM: warning "gatsby > @pmmmwh/[email protected]" has incorrect peer dependency "react-refresh@^0.8.2".
8:07:59 PM: warning "gatsby > @typescript-eslint/eslint-plugin > [email protected]" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "@reach/router@^1.1.1".
8:07:59 PM: warning "gatsby-plugin-favicon > [email protected]" has unmet peer dependency "webpack@^1.13.0 || ^2.0.0 || ^3.0.0 || ^4.0.0".
8:07:59 PM: warning "gatsby-plugin-sass > [email protected]" has unmet peer dependency "webpack@^3.0.0 || ^4.0.0".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "babel-plugin-styled-components@>1.5.0".
8:07:59 PM: warning "gatsby-transformer-inline-svg > gatsby > [email protected]" has unmet peer dependency "@types/react@^15.0.0 || ^16.0.0".
8:07:59 PM: warning "gatsby-transformer-inline-svg > gatsby > gatsby-cli > [email protected]" has unmet peer dependency "@types/react@>=16.8.0".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "webpack@^2.0.0 || ^3.0.0 || ^4.0.0".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "moment@^2.18.1".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "react-with-direction@^1.3.0".
8:07:59 PM: warning "react-dates > [email protected]" has unmet peer dependency "moment@>=1.6.0".
8:07:59 PM: warning "react-stripe-menu > react-scripts > @typescript-eslint/[email protected]" has unmet peer dependency "typescript@*".
8:07:59 PM: warning "react-stripe-menu > react-scripts > @typescript-eslint/[email protected]" has unmet peer dependency "typescript@*".
8:07:59 PM: warning "react-stripe-menu > react-scripts > @typescript-eslint/eslint-plugin > @typescript-eslint/[email protected]" has unmet peer dependency "typescript@*".
8:07:59 PM: warning " > [email protected]" has unmet peer dependency "eslint-plugin-react-hooks@^1.7.0".
8:09:58 PM: [4/4] Building fresh packages...
8:10:32 PM: Done in 237.08s.
8:10:32 PM: NPM modules installed using Yarn

When combined with netlify/build#1137 this leads to significantly increased build times.

[ehmicky]

This appears not to be related to the beta, so I am transferring to build-image.
An issue was filed in #432.

[verythorough]

Marking this as type: bug because it predates Build Plugins work.

[JGAntunes]

👋 Hey folks, I'm trying to pinpoint some of the reported issues around yarn usage and caching problems. There seem to be two big groups of reports:

• Monorepo support (yarn workspaces and lerna repos) - the directory structure still poses a problem to the way we cache things.
• For a regular "non-monorepo" project, yarn install re-installs all the packages when no changes were performed.

I'm assuming this report is about the latter. If so I'm having some trouble reproducing this, and maybe someone could help me out. @jlengstorf I know it was a while back but maybe you can tell me if my assumptions above are correct and if so, do you have any examples of this issue I can look into? I've setup a quick gatsby project from a template - https://github.com/JGAntunes/gatsby-yarn-netlify-test - it has a yarn.lock file, but after the first run to setup all the dependencies, further executions don't trigger whole yarn installs - i.e. https://app.netlify.com/sites/gatsby-yarn-test/deploys/60252f74ca3c3a0007ca0af2 - am I missing something here?

[JGAntunes]

I'm closing this in favour of #432 and #479 as I'm assuming this is a bug report about yarn usage with workspaces. Again do let me know if that is not the case and we can re-open it 🙏