yarn doesn't install packages properly, sometimes links to cache

[mgol]

Do you want to request a feature or report a bug?

A bug.

What is the current behavior?
yarn sometimes doesn't install packages correctly:

1. Some packages have almost empty directories, containin just the node_modules folder and that's it.
2. Some binaries link to the yarn cache instead of locally installed packages.

If the current behavior is a bug, please provide the steps to reproduce.

With the following package.json:

{
  "name": "test-package",
  "private": true,
  "dependencies": {
    "autoprefixer": "6.5.3",
    "babel-core": "6.20.0",
    "babel-loader": "6.2.9",
    "babel-plugin-lodash": "3.2.10",
    "babel-plugin-transform-runtime": "6.15.0",
    "babel-preset-es2015": "6.18.0",
    "babel-preset-es2016": "6.16.0",
    "babel-preset-es2017": "6.16.0",
    "chalk": "1.1.3",
    "css-loader": "0.26.1",
    "directory-tree": "1.1.1",
    "eslint": "3.12.0",
    "eslint-config-mgol": "0.0.34",
    "eslint-loader": "1.6.1",
    "extract-text-webpack-plugin": "1.0.1",
    "file-loader": "0.9.0",
    "html-webpack-harddisk-plugin": "0.0.2",
    "html-webpack-plugin": "2.24.1",
    "jquery": "3.1.1",
    "json-loader": "0.5.4",
    "koa": "1.2.4",
    "koa-compress": "1.0.9",
    "koa-etag": "2.1.1",
    "koa-fresh": "0.0.3",
    "koa-hbs": "0.9.0",
    "koa-router": "5.4.0",
    "koa-static-cache": "3.1.7",
    "lodash": "4.17.2",
    "node-sass": "3.13.1",
    "postcss-loader": "1.2.0",
    "sass-loader": "4.0.2",
    "semver": "5.3.0",
    "style-loader": "0.13.1",
    "supports-color": "3.1.2",
    "url-loader": "0.5.7",
    "webpack": "1.14.0"
  },
  "devDependencies": {
    "browser-sync": "2.18.2",
    "concurrently": "3.1.0",
    "nodemon": "1.11.0",
    "webpack-dev-middleware": "1.8.4",
    "webpack-hot-middleware": "2.13.2"
  }
}

Fire yarn cache clean && yarn and then ./node_modules/.bin/concurrently. You'll get a message "Error: Cannot find module 'rx'". rx is a dependency of the concurrently package so it's weird that it's not installed. But look into ./node_modules/concurrently - it only contains node_modules and none files. Moreover, ./node_modules/concurrently/node_modules only contains supports-color; no rx here but also not under ./node_modules/rx so it's complaining correctly.

Why does the command even start then, since the package folder is almost empty? Let's look into ./node_modules/.bin:

browser-sync@ -> ../../../../../../Library/Caches/Yarn/npm-browser-sync-2.18.2-3c101274c507e2f734dbce4e4b19bd3e124c3ea7/bin/browser-sync.js
concurrent@ -> ../../../../../../Library/Caches/Yarn/npm-concurrently-3.1.0-dc5ef0459090012604756668894c04b434ef90d1/src/main.js
concurrently@ -> ../../../../../../Library/Caches/Yarn/npm-concurrently-3.1.0-dc5ef0459090012604756668894c04b434ef90d1/src/main.js
eslint@ -> ../eslint/bin/eslint.js
node-sass@ -> ../node-sass/bin/node-sass
nodemon@ -> ../../../../../../Library/Caches/Yarn/npm-nodemon-1.11.0-226c562bd2a7b13d3d7518b49ad4828a3623d06c/bin/nodemon.js
semver@ -> ../semver/bin/semver
webpack@ -> ../webpack/bin/webpack.js

As you can see, some packages are taken from the yarn cache instead of the local location.

What is the expected behavior?

1. It should work.
2. None binaries should be linked to the yarn cache, everything should exist locally.

Please mention your node.js, yarn and operating system version.
Node 6.9.2, yarn 0.17.10 (installed via Homebrew), macOS 10.12.1 (16B2555)

[mgol]

If I merge devDependencies into dependencies, remove node_modules & yarn.lock and then re-try, everything seems to work fine.

[mgol]

It turned out I had NODE_ENV=production set in this terminal. Still, it was weird that it sort-of-installed devDependencies at all in that scenario instead of skipping them completely.

[joeyespo]

I'm seeing this too.

It seems directly related to NODE_ENV=production. Heroku sets NODE_ENV=production and also NPM_CONFIG_PRODUCTION=false to install devDependencies regardless of NODE_ENV.

Looks like this might be a regression of #2104.

[jeffnappi]

Also seeing this sporatically with our CircleCI builds - with v0.19.1

[tmarshall]

If I merge devDependencies into dependencies, remove node_modules & yarn.lock and then re-try, everything seems to work fine.

I was able to resolve it by just removing yarn.lock and re-installing dependencies