Yarn: Doesn't install correct dependencies for 1.3.2

Created on 3 Jan 2018  Β·  24Comments  Β·  Source: yarnpkg/yarn

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
Installation of packages fails when installing in production, Yarn 1.3.2 errors with:

Error: Cannot find module 'are-we-there-yet'

This does not happen with Yarn 0.18.1, where the installs complete successfully.

If the current behavior is a bug, please provide the steps to reproduce.
These are the relevant dependencies for a particular package that fails:

"dependencies": {
    "@meanie/angular-analytics": "^2.0.0",
    "@meanie/angular-api": "^3.0.1",
    "@meanie/angular-convert": "^2.0.1",
    "@meanie/angular-duplicate-requests-filter": "^2.0.0",
    "@meanie/angular-focus": "^2.0.0",
    "@meanie/angular-key-codes": "^2.0.0",
    "@meanie/angular-log": "^2.0.0",
    "@meanie/angular-modal": "^3.0.0",
    "@meanie/angular-storage": "^2.0.1",
    "@meanie/angular-store": "^2.0.1",
    "@meanie/angular-url": "^2.0.1",
    "angular": "1.6.8",
    "angular-animate": "1.6.8",
    "angular-cookies": "1.6.8",
    "angular-messages": "1.6.8",
    "angular-sanitize": "1.6.8",
    "angular-touch": "1.6.8",
    "angular-ui-router": "^0.4.2",
    "fastclick": "^1.0.6",
    "marked": "^0.3.9",
    "meanie-angular-form-controls": "^1.4.5",
    "moment": "^2.20.1",
    "ng-infinite-scroll": "^1.3.0",
    "raven-js": "^3.21.0"
  },
  "devDependencies": {
    "babel-polyfill": "^6.23.0",
    "babel-preset-es2015": "^6.24.0",
    "browser-sync": "^2.18.6",
    "chalk": "^1.1.3",
    "del": "^2.2.2",
    "git-rev-sync": "^1.8.0",
    "gulp": "git+https://github.com/gulpjs/gulp#4.0",
    "gulp-angular-templatecache": "^2.0.0",
    "gulp-autoprefixer": "^3.1.1",
    "gulp-babel": "^6.1.2",
    "gulp-concat": "^2.6.1",
    "gulp-csso": "^3.0.0",
    "gulp-file": "^0.4.0",
    "gulp-htmlclean": "^2.7.16",
    "gulp-inject": "^4.3.0",
    "gulp-ng-annotate": "^2.1.0",
    "gulp-ng-constant": "^1.1.0",
    "gulp-plumber": "^1.1.0",
    "gulp-preprocess": "^2.0.0",
    "gulp-remove-empty-lines": "^0.0.8",
    "gulp-remove-html-comments": "^1.0.1",
    "gulp-rename": "^1.2.2",
    "gulp-sass": "^3.1.0",
    "gulp-sourcemaps": "^2.4.1",
    "gulp-uglify": "^2.1.2",
    "gulp-wrapper": "^1.0.0",
    "http-proxy-middleware": "^0.17.4",
    "merge-stream": "^1.0.1",
    "npm-run-all": "^4.0.2",
    "mime": "^1.3.4",
    "netlify-cli": "^1.2.2",
    "replace-in-file": "^2.5.0",
    "rimraf": "^2.6.1",
    "s3-cli": "^0.13.0",
    "serve-static": "^1.12.1",
    "yargs": "^10.0.3"
  }

What is the expected behavior?
For the Yarn install to work.

Please mention your node.js, yarn and operating system version.
Node.JS 8.9.3, Yarn 1.3.2, the build was failing on Netlify VM's.

Ref #761

cat-bug high-priority triaged
Source
picture adamreisnz
👍3

Most helpful comment

Setting the flags did the trick for me πŸ‘

image

picture peduarte on 7 Feb 2018
👍3

All 24 comments

I was unable to reproduce using the same dependencies:

~/Projects/yarn-test πŸ’   cat package.json
{
  "name": "yarn-test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
"dependencies": {
    "@meanie/angular-analytics": "^2.0.0",
    "@meanie/angular-api": "^3.0.1",
    "@meanie/angular-convert": "^2.0.1",
    "@meanie/angular-duplicate-requests-filter": "^2.0.0",
    "@meanie/angular-focus": "^2.0.0",
    "@meanie/angular-key-codes": "^2.0.0",
    "@meanie/angular-log": "^2.0.0",
    "@meanie/angular-modal": "^3.0.0",
    "@meanie/angular-storage": "^2.0.1",
    "@meanie/angular-store": "^2.0.1",
    "@meanie/angular-url": "^2.0.1",
    "angular": "1.6.8",
    "angular-animate": "1.6.8",
    "angular-cookies": "1.6.8",
    "angular-messages": "1.6.8",
    "angular-sanitize": "1.6.8",
    "angular-touch": "1.6.8",
    "angular-ui-router": "^0.4.2",
    "fastclick": "^1.0.6",
    "marked": "^0.3.9",
    "meanie-angular-form-controls": "^1.4.5",
    "moment": "^2.20.1",
    "ng-infinite-scroll": "^1.3.0",
    "raven-js": "^3.21.0"
  },
  "devDependencies": {
    "babel-polyfill": "^6.23.0",
    "babel-preset-es2015": "^6.24.0",
    "browser-sync": "^2.18.6",
    "chalk": "^1.1.3",
    "del": "^2.2.2",
    "git-rev-sync": "^1.8.0",
    "gulp": "git+https://github.com/gulpjs/gulp#4.0",
    "gulp-angular-templatecache": "^2.0.0",
    "gulp-autoprefixer": "^3.1.1",
    "gulp-babel": "^6.1.2",
    "gulp-concat": "^2.6.1",
    "gulp-csso": "^3.0.0",
    "gulp-file": "^0.4.0",
    "gulp-htmlclean": "^2.7.16",
    "gulp-inject": "^4.3.0",
    "gulp-ng-annotate": "^2.1.0",
    "gulp-ng-constant": "^1.1.0",
    "gulp-plumber": "^1.1.0",
    "gulp-preprocess": "^2.0.0",
    "gulp-remove-empty-lines": "^0.0.8",
    "gulp-remove-html-comments": "^1.0.1",
    "gulp-rename": "^1.2.2",
    "gulp-sass": "^3.1.0",
    "gulp-sourcemaps": "^2.4.1",
    "gulp-uglify": "^2.1.2",
    "gulp-wrapper": "^1.0.0",
    "http-proxy-middleware": "^0.17.4",
    "merge-stream": "^1.0.1",
    "npm-run-all": "^4.0.2",
    "mime": "^1.3.4",
    "netlify-cli": "^1.2.2",
    "replace-in-file": "^2.5.0",
    "rimraf": "^2.6.1",
    "s3-cli": "^0.13.0",
    "serve-static": "^1.12.1",
    "yargs": "^10.0.3"
  }
}

~/Projects/yarn-test πŸ’   yarn --production
yarn install v1.3.2
info No lockfile found.
[1/4] πŸ”  Resolving packages...
warning [email protected]: This npm package 'angular-ui-router' has been renamed to '@uirouter/angularjs'. Please update your package.json. See https://ui-router.github.io/blog/uirouter-scoped-packages/
warning [email protected]: Package moved to @meanie/angular-form-controls
warning [email protected]: πŸ™Œ  Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!
warning gulp-angular-templatecache > gulp-header > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning gulp-autoprefixer > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning gulp-file > through2 > xtend > [email protected]:
warning gulp-htmlclean > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning gulp-remove-empty-lines > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning gulp-remove-html-comments > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning gulp-sass > [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
warning netlify-cli > netlify > [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
warning s3-cli > s3 > [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
[2/4] 🚚  Fetching packages...
[3/4] πŸ”—  Linking dependencies...
[4/4] πŸ“ƒ  Building fresh packages...
success Saved lockfile.
✨  Done in 18.16s.

Could you try running with --verbose? It might help point to the problem...

rally25rs picture rally25rs on 7 Jan 2018

Unfortunately, I don't have control over what flags yarn is launched with when Netlify runs the build afaik.

This is the full relevant log I can produce at this stage;

9:53:26 AM: Now using node v8.9.3 (npm v5.5.1)
9:53:26 AM: Using version v8.9.3 of node
9:53:26 AM: Using /opt/buildhome/.rvm/gems/ruby-2.1.2
9:53:26 AM: Installing yarn at version 1.3.2
9:53:27 AM: Installing Yarn!
9:53:27 AM: > Downloading tarball...
...
9:53:40 AM: > Verifying integrity...
9:53:40 AM: gpg:
9:53:40 AM: Signature made Thu 02 Nov 2017 04:44:10 PM UTC using RSA key ID FD2497F5
9:53:40 AM: gpg: Good signature from "Yarn Packaging <[email protected]>"
9:53:40 AM: gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
9:53:40 AM: Primary key fingerprint: 72EC F46A 56B4 AD39 C907  BBB7 1646 B01B 86E5 0310
     Subkey fingerprint: 6A01 0C51 6600 6599 AA17  F081 46C2 130D FD24 97F5
9:53:40 AM: > GPG signature looks good
9:53:40 AM: > Extracting to ~/.yarn...
9:53:41 AM: > Adding to $PATH...
9:53:41 AM: > We've added the following to your /opt/buildhome/.profile
> If this isn't the profile of your current shell then please add the following to your correct profile:

export PATH="$HOME/.yarn/bin:$PATH"
9:53:41 AM: > Successfully installed Yarn 1.3.2! Please open another terminal where the `yarn` command will now be available.
9:53:41 AM: Installing NPM modules using Yarn version 1.3.2
9:53:41 AM: yarn install v1.3.2
9:53:41 AM: [1/5] Validating package.json...
9:53:41 AM: [2/5] Resolving packages...
9:53:41 AM: [3/5] Fetching packages...
9:53:44 AM: info [email protected]: The platform "linux" is incompatible with this module.
9:53:44 AM: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
9:53:44 AM: [4/5] Linking dependencies...
9:53:50 AM: [5/5] Building fresh packages...
9:53:50 AM: error /opt/build/repo/node_modules/node-sass: Command failed.
Exit code: 1
Command: node scripts/install.js
Arguments: 
Directory: /opt/build/repo/node_modules/node-sass
Output:
module.js:538
    throw err;
    ^

Error: Cannot find module 'are-we-there-yet'
    at Function.Module._resolveFilename (module.js:536:15)
    at Function.Module._load (module.js:466:25)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/opt/build/repo/node_modules/npmlog/log.js:2:16)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
9:53:50 AM: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
9:53:50 AM: Error during Yarn install
9:53:53 AM: Build complete: exit code: 1

I made sure to clear the build cache etc. before deploying.

adamreisnz picture adamreisnz on 7 Jan 2018

You know what's weird about that... it's running the install script for node-sass but that is a transitive devDependency and shouldn't even be installed with --production. πŸ€” 

devDependencies -> gulp-sass -> node-sass
rally25rs picture rally25rs on 7 Jan 2018

Sorry, my bad! This of course is not run with the --production flag because it needs the dev deps to build the actual application. I was confused with the Heroku (backend) builds for a second...

I have updated the title and description of the issue, and will ask what other (if any) flags and parameters they use over at Netlify when kicking off the build, to see if that clarifies what's happening.

adamreisnz picture adamreisnz on 7 Jan 2018

cc @BYK. We discussed recently a known problem with transitive optional dependencies, could be related

bestander picture bestander on 8 Jan 2018

I think it might be. This is the build script Netlify uses, and from what I can see it uses the --ignore-optional flag.

adamreisnz picture adamreisnz on 9 Jan 2018
🎉2

Btw, https://github.com/yarnpkg/yarn/pull/5059 is the PR to fix --ignore-optional. I haven't had time to fix the latest review feedback just yet.

kimjoar picture kimjoar on 15 Jan 2018
👍1

@adamreisnz yeah, @kjbekkelund's patch should fix this then.

@kjbekkelund I'm not sure if we need another pass on that patch. I think we'll just merge it as it is and then try to address the larger problem separately.

BYK picture BYK on 15 Jan 2018
🎉1

@BYK That works for me. Been busy on a release here, so was planning to get back to it in a couple days.

kimjoar picture kimjoar on 15 Jan 2018

@kjbekkelund my latest comment was about simpler tests anyways. I'll try to see if I can do it myself and then merge the patch.

BYK picture BYK on 15 Jan 2018
👍2

Awesome thanks guys, let me know once it's released and I'll run some tests on Netlify again.

adamreisnz picture adamreisnz on 15 Jan 2018
👍1

@adamreisnz well, if you can check in https://8470-49970642-gh.circle-artifacts.com/0/yarnpkg/yarn-1.3.2.js and then add yarn-path "path/to/checked-in/file" to your .yarnrc file, you should be able to test the version with the patch applied.

BYK picture BYK on 15 Jan 2018
👍1

I'm not sure I have that level of control over the build process at Netlify

On Tue, Jan 16, 2018, 07:50 Burak Yiğit Kaya notifications@github.com
wrote:

@adamreisnz https://github.com/adamreisnz well, if you can check in
https://8470-49970642-gh.circle-artifacts.com/0/yarnpkg/yarn-1.3.2.js and
then add yarn-path "path/to/checked-in/file" to your .yarnrc file, you
should be able to test the version with the patch applied.

β€”
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/5152#issuecomment-357762236, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAd8QmYmAeW1gAkXBosEnUF8WlESerSZks5tK54BgaJpZM4RSH6Z
.

adamreisnz picture adamreisnz on 15 Jan 2018

Is there a new version going to be released soon? I feel really sad for still having to use yarn 0.18.2 on Netlify :(

adamreisnz picture adamreisnz on 6 Feb 2018

Hey folks. I'm also having this problem, also ran into it with Netlify.

@BYK I just tried your suggestion and it worked! πŸ‘Thank you. This was the end of quite a frustrating day.

lourd picture lourd on 6 Feb 2018

@adamreisnz, you can use the release candidate on Netlify, which has this fix, by making a netlify.toml file in your repo with at least these contents:

[build.environment]
  YARN_VERSION = "1.4.0"
lourd picture lourd on 6 Feb 2018

Thanks I'll give it a go. Didn't think specifying the version flag would work if the version hasn't officially been released et.

adamreisnz picture adamreisnz on 7 Feb 2018

1.4.0 doesn't seem to have fixed it.
Still getting errors for packages that have failed to install:

12:45:19 PM: yarn install v1.4.0
12:45:19 PM: [1/5] Validating package.json...
12:45:19 PM: [2/5] Resolving packages...
12:45:20 PM: [3/5] Fetching packages...
12:45:29 PM: info [email protected]: The platform "linux" is incompatible with this module.
12:45:29 PM: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
12:45:29 PM: [4/5] Linking dependencies...
12:45:32 PM: [5/5] Building fresh packages...
12:45:32 PM: error An unexpected error occurred: "/opt/build/repo/node_modules/node-sass: Command failed.\nExit code: 1\nCommand: sh\nArguments: -c node scripts/install.js\nDirectory: /opt/build/repo/node_modules/node-sass\nOutput:\nmodule.js:540\n    throw err;\n    ^\n\nError: Cannot find module 'wrappy'\n    at Function.Module._resolveFilename (module.js:538:15)\n    at Function.Module._load (module.js:468:25)\n    at Module.require (module.js:587:17)\n    at require (internal/module.js:11:18)\n    at Object.<anonymous> (/opt/build/repo/node_modules/inflight/inflight.js:1:76)\n    at Module._compile (module.js:643:30)\n    at Object.Module._extensions..js (module.js:654:10)\n    at Module.load (module.js:556:32)\n    at tryModuleLoad (module.js:499:12)\n    at Function.Module._load (module.js:491:3)".
12:45:32 PM: info If you think this is a bug, please open a bug report with the information provided in "/opt/build/repo/yarn-error.log".
12:45:32 PM: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
12:45:32 PM: Error during Yarn install

These are the dependencies I'm trying to install (all packages are up to date):

"dependencies": {
    "@meanie/angular-api": "^3.0.1",
    "@meanie/angular-convert": "^2.0.1",
    "@meanie/angular-duplicate-requests-filter": "^2.0.0",
    "@meanie/angular-focus": "^2.0.0",
    "@meanie/angular-form-controls": "^3.2.1",
    "@meanie/angular-google-maps": "^2.0.0",
    "@meanie/angular-key-codes": "^2.0.0",
    "@meanie/angular-log": "^2.0.0",
    "@meanie/angular-modal": "^3.1.0",
    "@meanie/angular-storage": "^2.0.1",
    "@meanie/angular-store": "^2.0.1",
    "@meanie/angular-url": "^2.0.1",
    "@uirouter/angularjs": "^1.0.13",
    "@uirouter/core": "^5.0.14",
    "angular": "^1.6.9",
    "angular-animate": "^1.6.9",
    "angular-cookies": "^1.6.9",
    "angular-messages": "^1.6.9",
    "angular-sanitize": "^1.6.9",
    "angular-touch": "^1.6.9",
    "moment": "^2.20.1",
    "raven-js": "^3.22.1"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-plugin-angularjs-annotate": "^0.8.2",
    "babel-plugin-syntax-async-functions": "^6.13.0",
    "babel-plugin-transform-regenerator": "^6.26.0",
    "babel-polyfill": "^6.26.0",
    "babel-preset-env": "^1.6.1",
    "babel-preset-es2015": "^6.24.1",
    "browser-sync": "^2.23.6",
    "chalk": "^2.3.0",
    "deepmerge": "^2.0.1",
    "del": "^3.0.0",
    "git-rev-sync": "^1.9.1",
    "gulp": "next",
    "gulp-angular-templatecache": "^2.2.0",
    "gulp-autoprefixer": "^4.1.0",
    "gulp-babel": "^7.0.0",
    "gulp-cached": "^1.1.1",
    "gulp-concat": "^2.6.1",
    "gulp-csso": "^3.0.1",
    "gulp-file": "^0.4.0",
    "gulp-htmlclean": "^2.7.16",
    "gulp-inject": "^4.3.0",
    "gulp-ng-constant": "^1.2.0",
    "gulp-plumber": "^1.2.0",
    "gulp-preprocess": "^2.0.0",
    "gulp-remove-empty-lines": "^0.1.0",
    "gulp-rename": "^1.2.2",
    "gulp-sass": "^3.1.0",
    "gulp-sourcemaps": "^2.6.4",
    "gulp-strip-comments": "^2.5.2",
    "gulp-uglify": "^3.0.0",
    "http-proxy-middleware": "^0.17.4",
    "merge-stream": "^1.0.1",
    "npm-run-all": "^4.1.2",
    "replace-in-file": "^3.1.0",
    "serve-static": "^1.13.1",
    "yargs": "^11.0.0"
  }

And to clarify, this works fine with 0.18.2 or when installing without the --ignore-optional flag on my dev machine.

adamreisnz picture adamreisnz on 7 Feb 2018

Aw bummer ☹️not sure why that is.

I also just heard back from one of the Netlify engineers who recommended this:

The setting we set is "--ignore-optional" (which is mentioned in that issue you linked to) and to override that, set a YARN_FLAGS Build Environment Variable (most easily on your build & deploy settings page, second configuration card - also possible in netlify.toml if you use that).

You should set it to "--no-ignore-optional" and I suspect your build will work better. We'll change the default shortly, but for the meantime, that should help your build succeed.

lourd picture lourd on 7 Feb 2018

Hmm, so is Netlify going to change the default flag?
Regardless, this still seems like a Yarn bug with resolving (optional) dependencies incorrectly.

adamreisnz picture adamreisnz on 7 Feb 2018

FYI: Netlify is going to change the default flag - I work there, and we are working on rolling that fix out in the next couple of days.

fool picture fool on 7 Feb 2018
🎉1

Setting the flags did the trick for me πŸ‘

image

peduarte picture peduarte on 7 Feb 2018
👍3

I'm unfortunately running into this problem still, even setting the YARN_FLAGS to --no-ignore-optional and various specific versions of yarn, 1.4.0, 1.3.2 etc, everything fails on netlify as it seems to be missing the semver module:

3:11:30 PM: > node build/build.js
3:11:30 PM: module.js:549
3:11:30 PM: throw err;
3:11:30 PM: ^
3:11:30 PM: Error: Cannot find module 'semver'

This build works perfectly locally, so I'm not sure what can be done if all the build flags are being set properly according to this thread.

I'm even having this problem with a completely empty vue-init webpack project.

alancwoo picture alancwoo on 8 Aug 2018

This is likely not a problem with yarn and should be brought up with the
netlify helpdesk. (says the guy who runs the netlify helpdesk)

On Wed, Aug 8, 2018 at 9:49 AM Alan Woo notifications@github.com wrote:

I'm unfortunately running into this problem still, even setting the
YARN_FLAGS to --no-ignore-optional and various specific versions of yarn,
1.4.0, 1.3.2 etc, everything fails on netlify as it seems to be missing
the semver module:

3:11:30 PM: > node build/build.js
3:11:30 PM: module.js:549
3:11:30 PM: throw err;
3:11:30 PM: ^
3:11:30 PM: Error: Cannot find module 'semver'

This build works perfectly locally, so I'm not sure what can be done if
all the build flags are being set properly according to this thread.

β€”
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/5152#issuecomment-411474854, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAN-oBAlYa1fr-W1mpm-ecGhS6zbuk7Xks5uOxZ8gaJpZM4RSH6Z
.

fool picture fool on 8 Aug 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

sanex3339 picture sanex3339  Β·  3Comments
ocolot picture ocolot  Β·  3Comments
chiedo picture chiedo  Β·  3Comments
mnpenner picture mnpenner  Β·  3Comments
baptistelebail picture baptistelebail  Β·  3Comments