Yarn: Yarn check [--integrity [--verbose]] fail and messages aren't helpful in the slightest

[taoeffect]

Do you want to request a feature or report a bug?

Bug.

If the current behavior is a bug, please provide the steps to reproduce.

In my repo I ran several commands, in succession: yarn check, then yarn check --integrity and then yarn check --integrity --verbose.

$ yarn check
yarn check v0.17.10
warning standard#eslint-config-standard-jsx#eslint@>=3.0.0 could be deduped from 3.12.2 to [email protected]
warning standard#eslint-config-standard#eslint@>=3.8.1 could be deduped from 3.12.2 to [email protected]
warning standard#eslint-plugin-react#eslint@^2.0.0 || ^3.0.0 could be deduped from 3.12.2 to [email protected]
warning standard#eslint-plugin-standard#eslint@>=3.0.0 could be deduped from 3.12.2 to [email protected]
warning sqlite3#node-pre-gyp#mkdirp@~0.5.1 could be deduped from 0.5.1 to [email protected]
warning sqlite3#node-pre-gyp#nopt@~3.0.6 could be deduped from 3.0.6 to [email protected]
warning sqlite3#node-pre-gyp#npmlog@^4.0.0 could be deduped from 4.0.1 to [email protected]
warning sqlite3#node-pre-gyp#rc@~1.1.6 could be deduped from 1.1.6 to [email protected]
warning sqlite3#node-pre-gyp#request@^2.75.0 could be deduped from 2.79.0 to [email protected]
warning sqlite3#node-pre-gyp#rimraf@~2.5.4 could be deduped from 2.5.4 to [email protected]
warning sqlite3#node-pre-gyp#semver@~5.3.0 could be deduped from 5.3.0 to [email protected]
warning sqlite3#node-pre-gyp#tar@~2.2.1 could be deduped from 2.2.1 to [email protected]
warning sqlite3#node-pre-gyp#tar-pack@~3.3.0 could be deduped from 3.3.0 to sqlite3#[email protected]
error shelljs#glob is wrong version: expected 7.0.6, got 7.1.1
error es3ify#esprima-fb is wrong version: expected 3001.1.0-dev-harmony-fb, got 3001.0001.0000-dev-harmony-fb
error sass-graph#glob is wrong version: expected 7.0.6, got 7.1.1
warning fsevents#node-pre-gyp#mkdirp@~0.5.1 could be deduped from 0.5.1 to [email protected]
warning fsevents#node-pre-gyp#nopt@~3.0.6 could be deduped from 3.0.6 to [email protected]
error fsevents#node-pre-gyp#npmlog@^4.0.0 doesn't satisfy found match of fsevents#[email protected]
warning fsevents#node-pre-gyp#rc@~1.1.6 could be deduped from 1.1.6 to [email protected]
error fsevents#node-pre-gyp#request@^2.75.0 doesn't satisfy found match of fsevents#[email protected]
error fsevents#node-pre-gyp#rimraf@~2.5.4 doesn't satisfy found match of fsevents#[email protected]
error fsevents#node-pre-gyp#semver@~5.3.0 doesn't satisfy found match of fsevents#[email protected]
warning fsevents#node-pre-gyp#tar@~2.2.1 could be deduped from 2.2.1 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#debug@~2.2.0 could be deduped from 2.2.0 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#fstream@~1.0.10 could be deduped from 1.0.10 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#fstream-ignore@~1.0.5 could be deduped from 1.0.5 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#rimraf@~2.5.1 could be deduped from 2.5.4 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#tar@~2.2.1 could be deduped from 2.2.1 to [email protected]
warning fsevents#node-pre-gyp#tar-pack#uid-number@~0.0.6 could be deduped from 0.0.6 to [email protected]
warning fsevents#tar-pack#once#wrappy@1 could be deduped from 1.0.2 to [email protected]
warning fsevents#tar-pack#readable-stream#buffer-shims@^1.0.0 could be deduped from 1.0.0 to [email protected]
warning fsevents#tar-pack#readable-stream#core-util-is@~1.0.0 could be deduped from 1.0.2 to [email protected]
warning fsevents#tar-pack#readable-stream#inherits@~2.0.1 could be deduped from 2.0.3 to [email protected]
warning fsevents#tar-pack#readable-stream#process-nextick-args@~1.0.6 could be deduped from 1.0.7 to [email protected]
warning fsevents#tar-pack#readable-stream#string_decoder@~0.10.x could be deduped from 0.10.31 to [email protected]
warning fsevents#tar-pack#readable-stream#util-deprecate@~1.0.1 could be deduped from 1.0.2 to [email protected]
info Found 30 warnings.
error Found 7 errors.

So I was like, "Wtf does that mean and how do I fix it?"

Not finding an answer to that question, I tried yarn check --integrity:

$ yarn check --integrity
yarn check v0.17.10
error Integrity hashes don't match, expected 0d2d5ed99a323c75f28f927eacdca204a7f608efc8e8c6c51fad33f05a40bc90 but got 126117d25ed334e75dc9d9e850da60dbf3a4503ef90f9a9b8547117c181db872
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

Oh, OK. Something's broken, but I have no idea what it is or what to do about it.

Maybe yarn check --integrity --verbose will be more helpful?

$ yarn check --integrity --verbose
yarn check v0.17.10
error Integrity hashes don't match, expected 0d2d5ed99a323c75f28f927eacdca204a7f608efc8e8c6c51fad33f05a40bc90 but got 126117d25ed334e75dc9d9e850da60dbf3a4503ef90f9a9b8547117c181db872
verbose Error: Found 1 errors.
    at Object.<anonymous> (/usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/lib/cli/commands/check.js:179:13)
    at Generator.next (<anonymous>)
    at step (/usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

😞

What is the expected behavior?

For Yarn to speak plain English and tell me what's going on and how to fix it.

1. First, yarn check and its errors need to give some hint as to how to fix them.
2. Secondly, yarn check --integrity should probably be done by default on every yarn command. Didn't this project say it was supposed to be "secure"? But it's not providing even the most basic level of security (checksum verification) unless a special secret command that I just stumbled upon is run. That's pretty unexpected behavior. The docs make it sound like it does integrity checks by default, but apparently it doesn't.
3. Finally, if something goes wrong, Yarn should tell me what it actually is. I tried searching for the hashes it gave me (both of them) in the yarn.lock file and neither of them was there. WTF. How am I supposed to fix this problem at this point? Yarn should tell me exactly what to do.

Please mention your node.js, yarn and operating system version.

$ npm version
{ npm: '3.10.9',
  ares: '1.10.1-DEV',
  cldr: '30.0.2',
  http_parser: '2.7.0',
  icu: '58.1',
  modules: '51',
  node: '7.2.1',
  openssl: '1.0.2j',
  tz: '2016g',
  unicode: '9.0',
  uv: '1.10.1',
  v8: '5.4.500.44',
  zlib: '1.2.8' }

Mac OS X — 10.11.6.

Yarn v0.17.10. I would have done this with the latest version, v0.18.1, if that version was available on Homebrew, but it is not.

[6zz]

how do I resolve the "could be deduped..." warnings? I can't update the yarn.lock file directly right?

[spacegoing]

same issue here:

 warning "\u001b[2mchokidar#\u001b[22mfsevents#node-pre-gyp@^0.6.29"
 could be deduped from "0.6.32" to "[email protected]"

Would please some body answer:

• What does the word deduped mean?
• Why there are so many rubbish characters in the warning message?
• How to fix this?

[kevingelion]

Would appriciate some answers to the questions asked above!

[bhoule]

The chokidar#fsevents#node-pre-gyp@^0.6.29 could be deduped from 0.6.33 to [email protected] warning is a result of _node-pre-gyp_ being a bundled dependency of _fsevents_.

[Ridermansb]

I have an similar issue..

yarn check v0.20.3
error Integrity hashes don't match, expected "132aa146372c295e87a48cc37eec4cad3f21c1f7cff4046cadd5dd3e2b05c1be" but got "bc557edc7b8680dfbad1e2c711bd6dae3e87e057cc5f84a13f671008ed6e2515"
verbose Error: Found 1 errors.
    at MessageError (/Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/errors.js:8:5)
    at /Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/cli/commands/check.js:167:13
    at next (native)
    at step (/Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command

How can I fix this?

[saarons]

The fix I've been able to find comes from manually editing the yarn.lock file.

Let's say for example you have the following entries:

lodash-es@^4.17.3:
  version "4.17.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.17.4.tgz#dcc1d7552e150a0640073ba9cb31d70f032950e7"

lodash-es@^4.2.1:
  version "4.16.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.16.4.tgz#4dc3e2cf33a8c343028aa7f7e06d1c9697042599"

This can be manually edited down to:

lodash-es@^4.2.1, lodash-es@^4.17.3:
  version "4.17.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.17.4.tgz#dcc1d7552e150a0640073ba9cb31d70f032950e7"

Don't know how it got in this state but running yarn check again comes out clean.

[taoeffect]

Here's how it behaves with the latest 0.23.2 version:

W.T.F.

EDIT: created a separate issue for this related issue: #3242

[Artoria2e5]

@spacegoing Why there are so many rubbish characters in the warning message?

These are ANSI color codes. What kind of terminal are you using here? Is is pre-Windows 10 cmd (old conhost)?

[spacegoing]

@Artoria2e5 I'm using osx's built-in terminal

[bestander]

This are some fair points, we are teaching Yarn proper English a little bit at a time :)

The integrity checks became more silent if they are not actionable and they do run by default when you install.

The warnings warning standard#eslint-config-standard-jsx#eslint@>=3.0.0 could be deduped from 3.12.2 to [email protected] I think got fixed in 0.26.

But if they are not please send a PR!

[Subtletree]

Was seeing the same thing:
yarn v0.27.5

yarn check

warning "ember-wormhole#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "[email protected]"
warning "ember-ajax#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "[email protected]"
warning "ember-chrome-devtools#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "[email protected]"
warning "ember-cli-moment-shim#ember-cli-babel@^6.6.0" could be deduped from "6.8.2" to "[email protected]"
warning "ember-cli-qunit#ember-cli-babel@^6.8.1" could be deduped from "6.8.2" to "[email protected]"
... //many more

yarn.lock

ember-cli-babel@^6.0.0, ember-cli-babel@^6.0.0-beta.4, ember-cli-babel@^6.0.0-beta.7, ember-cli-babel@^6.1.0, ember-cli-babel@^6.3.0, ember-cli-babel@^6.4.1, ember-cli-babel@^6.6.0:
  version "6.6.0"
...
ember-cli-babel@^6.7.0:
  version "6.7.1"
...
ember-cli-babel@^6.8.0, ember-cli-babel@^6.8.2:
  version "6.8.2"
...
ember-cli-babel@^6.8.1:
  version "6.8.1"

Ended up having to recreate the lock file which fixed both problems

[mjstelly]

Which process is correct? I ran yarn check --integrity with this result:

yarn check v1.3.2
success Folder in sync.
✨  Done in 0.19s.

But running yarn check provides warnings:

yarn check v1.3.2
warning "sane#fsevents#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "[email protected]"
warning "@firebase/firestore#grpc#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "[email protected]"
warning "eslint-config-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
warning "eslint-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
warning "eslint-plugin-class-property#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
info Found 5 warnings.
success Folder in sync.
✨  Done in 3.90s.

Which one is to be believed?

[bestander]

—integrity does a light check that verifies that no one touched
node_modules after yarn installed it.

Regular check runs Yarn’s resolution mechanism that checks that the
installation tree matches yarn.lock.
The warnings indicate that something is not optimal but installation is
still correct

On Thu, Jan 18, 2018 at 5:33 AM Michael Stelly [e-mail]
wrote:

Which process is correct? I ran yarn check --integrity with this result:

yarn check v1.3.2
success Folder in sync.
✨ Done in 0.19s.

But running yarn check provides warnings:

yarn check v1.3.2
warning "sane#fsevents#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "[email protected]"
warning "@firebase/firestore#grpc#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "[email protected]"
warning "eslint-config-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
warning "eslint-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
warning "eslint-plugin-class-property#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "[email protected]"
info Found 5 warnings.
success Folder in sync.
✨ Done in 3.90s.

Which one is to be believed?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/2287#issuecomment-358518614, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWPTh_-WBrR0n66gXuV34lIo7pXKpks5tLq2DgaJpZM4LQS0w
.

[robbie-hinman]

So... yarn check provides 13 warnings of packages that 'could be deduped' and I run yarn dedupe and am told yarn install will already dedupe and then I run yarn install and then it says I'm good and then run yarn check and I get 13 warnings... and yarn.lock says not to manually edit the file.

If it was just warnings I wouldn't be so concerned but our app recently and mysteriously broke and the only culprit is a ton of changes to a checked in yarn.lock file...

[bestander]

Often dedupe warnings are not really actionable by end users because this is in control of package authors.
@robbie-hinman, my bet is that your break is not linked to the duplicate warnings you see.
Yarn should clean this up and warn only on actionable cases, unfortunately the core team is busy with so many things it will take a lot of time till we get to fixing this.
Help from someone from the community will be very much appreciated

[halo]

I noticed that the yarn check output posted here is usually somewhat helpful:

• error ... is wrong version: expected ..., got ...
• error ... doesn't satisfy found match of ...
• error Integrity hashes don't match, expected ...

In my case it doesn't give me any hint as to what went wrong. I just run yarn install and then it works again for an hour or so until it breaks again with the following output 🤪

$ yarn check --integrity --verbose
yarn check v1.3.2
[...]
verbose 0.475 Checking for configuration file "/Users/.yarnrc".
verbose 0.477 current time: 2018-02-08T10:41:55.530Z
warning Integrity check: Flags don't match
error Integrity check failed
verbose 0.601 Error: Found 1 errors.
    at /path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:57476:13
    at Generator.next (<anonymous>)
    at step (/path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:92:30)
    at /path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:103:13
    at <anonymous>
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

Of course I hope this is not related to my setup but I don't think I do anything special. Just a normal Rails app using yarn.

Just to be clear: I'm not complaining, I'm just adding my output because it appears to be unique. The yarn team are heroes altogether!

[bestander]

Thanks for feedback
Here is the error https://github.com/yarnpkg/yarn/blob/f69cdda63ab6f36bdd21c6f4ab838e7b6200b1d6/__tests__/commands/check.js#L213, here is the text https://github.com/yarnpkg/yarn/blob/1b4b318355115cff5e58f128c53709b37b6ed28e/src/reporters/lang/en.js#L360

A PR to fix this would make Yarn even better.

[kaiyoma]

Haven't seen any recent activity here, but wanted to report that we have been running into this issue for quite a while now. In our project, yarn check spits out anywhere between 50 and 100 warnings that look like this:

...
warning "jest-runtime#babel-core#lodash@^4.17.4" could be deduped from "4.17.10" to "[email protected]"
warning "jest-snapshot#jest-diff#chalk@^2.0.1" could be deduped from "2.4.1" to "[email protected]"
warning "jest-config#jest-jasmine2#chalk@^2.0.1" could be deduped from "2.4.1" to "[email protected]"
...

There's no explanation of what's wrong (if anything) or what to do. Instead, legitimate warnings/errors are obfuscated by a deluge of output that isn't helpful and can't be suppressed.

[simonbuchan]

I've noticed file:../some-dir and link: deps will cause the "could be deduped" errors, presumably because in the first case, yarn incorrectly bundles the target dir's node_modules, and in the second because check doesn't know what to do with symlinks (probably just ignore them, right?)

If I get time to verify these soon I'll open bugs for them, but for now this might let people know what's up.