Yarn: Upgrade command fails with Yarn 1.5.1/1.6.0

[ahamana]

Current Outcome

At my pacage.json, the upgrade command fails with Yarn 1.5.1/1.6.0. In case of 1.3.2, not so.
The error log is different every time, but it always fails.
* The install command has no problem.

[ package.json ]
{ "main": "gulpfile.js", "license": "Unlicense", "devDependencies": { "@types/autosize": "3.0.6", "@types/bootstrap": "3.3.38", "@types/clipboard": "2.0.0", "@types/eonasdan-bootstrap-datetimepicker": "4.17.26", "@types/jquery": "3.3.1", "@types/jquery.blockui": "0.0.28", "@types/purl": "2.3.30", "del": "3.0.0", "fancy-log": "1.3.2", "gulp": "^4.0.0", "gulp-autoprefixer": "5.0.0", "gulp-clean-css": "3.9.3", "gulp-csscomb": "3.0.8", "gulp-debug": "3.2.0", "gulp-filter": "5.1.0", "gulp-htmlmin": "4.0.0", "gulp-if": "2.0.2", "gulp-imagemin": "4.1.0", "gulp-notify": "3.2.0", "gulp-plumber": "1.2.0", "gulp-rename": "1.2.2", "gulp-sass": "4.0.1", "gulp-token-replace": "1.1.2", "gulp-typedoc": "2.2.0", "gulp-typescript": "4.0.2", "gulp-uglify": "3.0.0", "gulp-zip": "4.1.0", "lazypipe": "1.0.1", "main-bower-files": "2.13.1", "merge2": "1.2.1", "object-merge": "2.5.1", "path-exists": "3.0.0", "typedoc": "0.11.1", "typescript": "2.8.1", "uglify-save-license": "0.4.1" }, "dependencies": { "@allmarkedup/purl": "allmarkedup/purl#2.3.2", "@bower_components/add-to-homescreen": "cubiq/add-to-homescreen#3.2.3", "@bower_components/autosize": "jackmoore/autosize#4.0.1", "@bower_components/blockUI": "malsup/blockui#*", "@bower_components/bootstrap": "twbs/bootstrap#3.3.7", "@bower_components/bootstrap-select": "silviomoreto/bootstrap-select#1.12.4", "@bower_components/bootstrap-show-password": "wenzhixin/bootstrap-show-password#1.1.2", "@bower_components/clipboard": "zenorocha/clipboard.js#2.0.0", "@bower_components/eonasdan-bootstrap-datetimepicker": "Eonasdan/bootstrap-datetimepicker#4.17.47", "@bower_components/font-awesome": "FortAwesome/Font-Awesome#5.0.10", "@bower_components/instantclick": "dieulot/instantclick#3.1.0", "@bower_components/jquery": "jquery/jquery-dist#3.3.1", "@bower_components/jquery-i18n-properties": "jquery-i18n-properties/jquery-i18n-properties#1.2.7", "@bower_components/jquery-toast-plugin": "kamranahmedse/jquery-toast-plugin#1.3.2", "@bower_components/moment": "moment/moment#2.22.1", "@bower_components/moment-timezone": "moment/moment-timezone#0.5.14", "@bower_components/pretty-checkbox": "lokesh-coder/pretty-checkbox#3.0.3", "@bower_components/purl": "allmarkedup/purl#2.3.2", "Font-Awesome": "FortAwesome/Font-Awesome#5.0.10", "add-to-homescreen": "cubiq/add-to-homescreen#3.2.3", "autosize": "jackmoore/autosize#4.0.1", "blockui": "malsup/blockui#*", "bootstrap": "twbs/bootstrap#3.3.7", "bootstrap-select": "silviomoreto/bootstrap-select#1.12.4", "bootstrap-show-password": "wenzhixin/bootstrap-show-password#1.1.2", "clipboard": "zenorocha/clipboard.js#2.0.0", "eonasdan-bootstrap-datetimepicker": "Eonasdan/bootstrap-datetimepicker#4.17.47", "instantclick": "dieulot/instantclick#3.1.0", "jquery": "jquery/jquery-dist#3.3.1", "jquery-i18n-properties": "jquery-i18n-properties/jquery-i18n-properties#1.2.7", "jquery-toast-plugin": "kamranahmedse/jquery-toast-plugin#1.3.2", "moment": "moment/moment#2.22.1", "moment-timezone": "moment/moment-timezone#0.5.14", "pretty-checkbox": "lokesh-coder/pretty-checkbox#3.0.3" }, "engines": { "yarn": ">= 1.0.0" }, "scripts": { "postinstall": "node -e \"try { require('fs').symlinkSync(require('path').resolve('node_modules/@bower_components'), 'bower_components', 'junction') } catch (e) { }\"" } }

Environment

• Node.js Version: 8.11.1
• Yarn: 1.5.1/1.6.0
• Operating System: Windows 10/Amazon Linux(release 2017.09)

[rally25rs]

What error are you getting?

[ahamana]

For example, as follows.

Yarn version: 
  1.6.0

Node version: 
  8.11.1

Platform: 
  win32 x64

Trace: 
  TypeError: Cannot read property '_reference' of undefined
      at _loop2 (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:50845:36)
      at PackageLinker.resolvePeerModules (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:50883:19)
      at PackageLinker.<anonymous> (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:50934:12)
      at Generator.next (<anonymous>)
      at step (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:98:30)
      at C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:116:14
      at new Promise (<anonymous>)
      at new F (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:23469:28)
      at PackageLinker.<anonymous> (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:95:12)
      at PackageLinker.init (C:\Users\Administrator\git\foo\web\.gradle\yarn\yarn-v1.6.0\node_modules\yarn\lib\cli.js:50939:21)

[BYK]

Feels like a corrupted cache to me. Have you tried doing yarn cache clean?

[rally25rs]

I can reproduce it actually... something weird going on with a package where it thinks it has a peerDep on undefined but I haven't quite figured out why... in the process of debugging...

[rally25rs]

somehow resolver.getAllInfoForPackageName('bootstrap') is returning two copies of bootstrap, and undefined 😕

---

_update 1_

Inside package-resolver, this.patternsByPackage["bootstrap"] contains

[
  "bootstrap@^3.3",
  "twbs/bootstrap#3.3.7",
  "bootstrap@^3.3",
  "bootstrap@^3.3",
  "bootstrap@^3.3",
  "bootstrap@twbs/bootstrap#3.3.7"
]

but this.patterns["twbs/bootstrap#3.3.7"] is undefined

so somehow that pattern is in one list but not the other.

The only place I see twbs in the yarn.lock file is:

"@bower_components/bootstrap@twbs/bootstrap#3.3.7", bootstrap@twbs/bootstrap#3.3.7:
  version "3.3.7"
  resolved "https://codeload.github.com/twbs/bootstrap/tar.gz/0b9c4a4007c44201dce9a6cc1a38407005c26c86"

---

_update 2_

Tracked this back to the bug...

package-resolver's addPattern() can push the same name multiple times:

  addPattern(pattern: string, info: Manifest) {
    this.patterns[pattern] = info;

    const byName = (this.patternsByPackage[info.name] = this.patternsByPackage[info.name] || []);
    byName.push(pattern);
  }

but the removePattern() will only remove the first occurrence:

  removePattern(pattern: string) {
    const pkg = this.patterns[pattern];
    if (!pkg) {
      return;
    }

    const byName = this.patternsByPackage[pkg.name];
    if (!byName) {
      return;
    }

    byName.splice(byName.indexOf(pattern), 1);
    delete this.patterns[pattern];
  }

For this set of packages, when removePattern is called, it removes the pattern twbs/bootstrap#3.3.7 from the array:

[ 'bootstrap@^3.3',
  'twbs/bootstrap#3.3.7',
  'twbs/bootstrap#3.3.7',
  'bootstrap@^3.3',
  'bootstrap@^3.3',
  'bootstrap@^3.3',
  'bootstrap@twbs/bootstrap#3.3.7' ]

which leaves an occurrence of it still in the array which messes up yarn later on.

I think the fix for this would be to prevent addPackage from adding duplicates (which would reduce overall looping and redundant data-lookups over that array as well).

@BYK does that sound right?

If so, I'll work up a PR some time today.

[rally25rs]

@ahamana I made a PR that you can test out if you want. Thanks for reporting this! 🎉

[ahamana]

@rally25rs Many thanks for fixing the issue quickly!