yarn install --production is broken, installs plenty of devDependencies

[Rush]

I'm running yarn 0.16.1. Given the following package.json, yarn will install wrong dependencies:

{
  "name": "yarn-test",
  "version": "1.0.0",
  "private": true,
  "scripts": {
  },
  "dependencies": {
    "babel-plugin-transform-async-to-module-method": "6.8.0",
    "babel-preset-es2015-node6": "0.2.0",
    "babel-register": "6.5.0",
    "bluebird": "3.1.1",
    "body-parser": "~1.12.4",
    "bugsnag": "^1.6.5",
    "connect-gzip-static": "^1.0.0",
    "continuation-local-storage": "^3.1.4",
    "cookie-parser": "~1.3.5",
    "debug": "~2.2.0",
    "del": "^1.2.1",
    "express": "4.14.0",
    "filtrex": "0.5.4",
    "glob": "^5.0.10",
    "html-to-text": "^1.3.2",
    "http-proxy": "^1.11.2",
    "juice": "^1.4.2",
    "marked": "0.3.6",
    "md5": "^2.0.0",
    "memoizee": "^0.3.8",
    "messageformat": "0.3.1",
    "moment": "~2.10.6",
    "morgan": "~1.5.3",
    "ms": "^0.7.1",
    "multer": "~0.1.8",
    "node-watch": "^0.3.4",
    "pug": "2.0.0-beta6",
    "recursive-merge": "^1.2.0",
    "request-promise": "^1.0.0",
    "requirejs": "^2.1.17",
    "send": "^0.13.0",
    "serve-favicon": "~2.2.1",
    "wkhtmltopdf": "^0.1.5",
    "xcase": "^1.0.6"
  },
  "devDependencies": {
    "babel": "6.5.0",
    "babel-eslint": "5.0.0",
    "babel-plugin-transform-es2015-modules-commonjs": "6.5.0",
    "babel-plugin-transform-runtime": "6.15.0",
    "babel-preset-es2015": "6.5.0",
    "chai": "^2.3.0",
    "compressible": "^2.0.5",
    "eslint": "2.2.0",
    "fetch": "1.1.0",
    "gulp": "^3.9.0",
    "gulp-babel": "^6.1.1",
    "gulp-concat": "^2.6.0",
    "gulp-gzip": "^1.2.0",
    "gulp-if": "^1.2.5",
    "gulp-modify": "^0.1.1",
    "gulp-ng-annotate": "^1.1.0",
    "gulp-pug": "3.0.4",
    "gulp-rename": "^1.2.2",
    "gulp-rev": "^6.0.1",
    "gulp-sourcemaps": "^1.5.2",
    "gulp-uglify": "^1.3.0",
    "js-yaml": "^3.3.1",
    "jspm": "0.16.45",
    "mime": "^1.3.4",
    "minimist": "^1.2.0",
    "mocha": "^2.2.5",
    "ng-annotate": "^1.0.1",
    "run-sequence": "^1.1.3",
    "should": "^7.0.1"
  }
}

npm install --production yields 71M of node_modules
yarn install --production yields 340M of node_modules - clearly wrong

To reproduce:

mkdir -p test-yarn
cd test-yarn
wget http://x.rushbase.net/e089e40357ded1a4fff61ce528e5cc20297464b3/package.json
yarn install --production
# repeat again with npm install --production

[nrser]

i'm seeing the same issue, also with yarn 0.16.1, which is breaking my production builds

[samccone]

Thanks for the report!

[AlicanC]

If you actually hack yarn why to work with --production, it can't find a reason why a devDep should be installed.

I have tried to fix this, but I couldn't make sense of the code.

With --production, Install.p.init() does not immediately drop devDeps, but instead sets their visibility to ENVIRONMENT_IGNORE so your devDeps go through the whole installation process. During this process, something probably overwrites or just fails to detect ENVIRONMENT_IGNORE.

Why devDeps aren't completely dropped with --production? To still be able to generate a complete yarn.lock?

If that is the case, then isn't it pretty unnecessary to do all this work for production environments? Could we make --production require a yarn.lock and use it to skip the whole resolve step so --production is lightning fast?

[Rush]

Why devDeps aren't completely dropped with --production? To still be able to generate a complete yarn.lock?

If that is the case, then isn't it pretty unnecessary to do all this work for production environments? Could we make --production require a yarn.lock and use it to skip the whole resolve step so --production is lightning fast?

This would be a perfect solution in my workflow.

[sheerun]

yarn install --production doesn't even work on yarn package itself. How to reproduce:

1. Download latest yarn package from npm registry, unpack it
2. run yarn install --production --ignore-scripts --pure-lockfile --ignore-engines

See following output:

yarn install v0.16.1
[1/4] 🔍  Resolving packages...
error "/Users/sheerun/Source/npm-packer/dist/scripts/eslint-rules" doesn't exist.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

While clearly eslint-rules is listed as devDependencies. So it even fails on validation.

[tanx]

Perhaps a duplicate of https://github.com/yarnpkg/yarn/issues/761 ?

[psylence303]

Same happening for me, 'yarn install --production' tries to find a directory that is listed in 'devDependencies' and 'npm i --production' was not doing this.

[troydemonbreun]

Is it just me, or does the OP's issue seem to be that it is installing too much, not that it is expecting deps that were not installed (per #761)? @Rush would you confirm if #761 is a dupe for your issue?

[sheerun]

@troydemonbreun It's correlated

[troydemonbreun]

FYI, #1739 landed, which was a fix for #761 (don't have time at the moment to verify if it fixes OP's issue)

[leschekfm]

With the current yarn version (0.17.6) and the provided package.json I can still see a discrepancy between npm and yarn. So it seems that this issue wasn't fixed by the other issues

[sebmck]

Fixed via #2116.

[kyeotic]

@kittens I am still seeing this behavior with the 0.20.0 release

[leschekfm]

I can see the issue still too. I renamed the node_modules folders after installation for comparison.

leschekfm@lesubunew:~/projects/temp$ npm --version
4.2.0
leschekfm@lesubunew:~/projects/temp$ yarn --version
0.20.0
leschekfm@lesubunew:~/projects/temp$ du -hs node_modules_yarn/
219M    node_modules_yarn/
leschekfm@lesubunew:~/projects/temp$ du -hs node_modules_npm/
70M node_modules_npm/

[PatrickKing]

+1 for this, with Yarn 0.20.3

Edited to add: on at least one occasion, I have seen Yarn fail to install a production dependency with --prod as well.

[dwmkerr]

Same issue for me, causes a lot of problems with my build as dev dependencies contain things like custom linting rules which require SSH access to internal repos...

[isiahmeadows]

Still seeing it myself. @kittens Could this be re-opened?

Edit: Yarn v0.23.2

[SimenB]

@isiahmeadows there was a fix in 0.23.1 (#2921), are you on latest?

EDIT: You are 😭 What do you get with yarn --prod && yarn check --verify-tree --prod?

[isiahmeadows]

@SimenB I edited my comment with my version (v0.23.2).

[isiahmeadows]

In my case, I have 0 dependencies and several dev dependencies. So anything installing to node_modules is a bug.

[SimenB]

Oh, fancy

[SimenB]

Is this a case of doing yarn && yarn test && yarn --force not cleaning up, or a plain yarn --prod from non-existent node_modules installing lots of stuff?

[isiahmeadows]

@SimenB Here's a 100% repro on macOS:

1. Create this package.json in a clean directory:

   {
     "devDependencies": {
       "eslint": "*"
     }
   }
   

2. Run yarn --prod, optionally with --pure-lockfile.

3. There should be 3 entries installed in node_modules:

   • [email protected] from eslint, eslint#escope, and eslint#esquery
   • [email protected] from eslint#inquirer#readline2
   • [email protected] from eslint#inquirer#readline2#is-fullwidth-code-point

[isiahmeadows]

You can also repro this with "electron-download": "*", in which [email protected] is installed because of jsprim and sshpk depended on by electron-download#nugget#request#http-signature.

[isiahmeadows]

@SimenB

Is this a case of doing yarn && yarn test && yarn --force not cleaning up, or a plain yarn --prod from non-existent node_modules installing lots of stuff?

It's the latter case.

[Diokuz]

Have same problems here: caniuse-db (and some other packages) installed from nowhere when yarn --production. But they are in devDeps.

Could it be reopened? Or there is another opened issue?

yarn --version
0.24.6

[miukki]

how to install only production deps ? (ignore devDep) any solution for now ?