Yarn: Yarn pulling wrong version of dependency inside kibana repo

[princesszelda]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?

When running command
yarn add [email protected] --exact

yarn tries to pull the wrong version of source-map, pulling version 0.7.3
The output is:

yarn add v1.7.0
info No lockfile found.
[1/5] Validating package.json...
[2/5] Resolving packages...
warning glob > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning glob-all > glob > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning [email protected]: Jade has been renamed to pug, please install the latest version of pug instead of jade
warning jade > [email protected]: Please update to at least constantinople 3.1.1
warning jade > [email protected]: Deprecated, use jstransformer
warning [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning request > [email protected]: Use uuid module instead
warning grunt > [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
warning grunt-cli > findup-sync > glob > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning image-diff > buffered-spawn > [email protected]: cross-spawn no longer requires a build toolchain, use it instead!
warning jest > jest-cli > istanbul-api > [email protected]: 1.2.0 should have been a major version bump
warning karma-coverage > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning karma-coverage > istanbul > fileset > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning load-grunt-config > cson > [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
warning load-grunt-config > cson > cson-parser > [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
warning sinon > [email protected]: This package is unmaintained. Use @sinonjs/formatio instead
warning [email protected]: SuperTest 2.0+ supports promises natively; use that instead!
[3/5] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
error [email protected]: The engine "node" is incompatible with this module. Expected version ">= 8".
error Found incompatible module
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

If the current behavior is a bug, please provide the steps to reproduce.

This happens inside the kibana repository. When you clone kibana and checkout version 5.6.10, the error occurs. The package.json file lists source-map 0.5.6 and it still tries to pull 0.7.3

This is the content in the package.json file. source-map in devDependencies is listed as 0.5.6

{
  "name": "kibana",
  "description": "Kibana is an open source (Apache Licensed), browser based analytics and search dashboard for Elasticsearch. Kibana is a snap to setup and start using. Kibana strives to be easy to get started with, while also being flexible and powerful, just like Elasticsearch.",
  "keywords": [
    "kibana",
    "elasticsearch",
    "logstash",
    "analytics",
    "visualizations",
    "dashboards",
    "dashboarding"
  ],
  "private": false,
  "version": "5.6.10",
  "branch": "5.6",
  "build": {
    "number": 8467,
    "sha": "6cb7fec4e154faa0a4a3fee4b33dfef91b9870d9"
  },
  "homepage": "https://www.elastic.co/products/kibana",
  "bugs": {
    "url": "http://github.com/elastic/kibana/issues"
  },
  "license": "Apache-2.0",
  "author": "Rashid Khan <[email protected]>",
  "contributors": [
    "Chris Cowan <[email protected]>",
    "Court Ewing <[email protected]>",
    "Jim Unger <[email protected]>",
    "Joe Fleming <[email protected]>",
    "Jon Budzenski <[email protected]>",
    "Juan Thomassie <[email protected]>",
    "Khalah Jones-Golden <[email protected]>",
    "Lee Drengenberg <[email protected]>",
    "Lukas Olson <[email protected]>",
    "Matt Bargar <[email protected]>",
    "Nicolás Bevacqua <[email protected]>",
    "Shelby Sturgis <[email protected]>",
    "Spencer Alger <[email protected]>",
    "Tim Sullivan <[email protected]>"
  ],
  "scripts": {
    "test": "grunt test",
    "test:dev": "grunt test:dev",
    "test:quick": "grunt test:quick",
    "test:browser": "grunt test:browser",
    "test:ui": "grunt test:ui",
    "test:ui:server": "grunt test:ui:server",
    "test:ui:runner": "echo 'use `node scripts/functional_test_runner`' && false",
    "test:server": "grunt test:server",
    "test:coverage": "grunt test:coverage",
    "test:visualRegression": "grunt test:visualRegression:buildGallery",
    "checkLicenses": "grunt licenses",
    "build": "grunt build",
    "release": "grunt release",
    "start": "sh ./bin/kibana --dev",
    "precommit": "grunt precommit",
    "karma": "karma start",
    "elasticsearch": "grunt esvm:dev:keepalive",
    "lint": "echo 'use `node scripts/eslint`' && false",
    "lintroller": "echo 'use `node scripts/eslint --fix`' && false",
    "makelogs": "echo 'use `node scripts/makelogs`' && false",
    "mocha": "echo 'use `node scripts/mocha`' && false",
    "sterilize": "grunt sterilize",
    "uiFramework:start": "grunt uiFramework:start",
    "uiFramework:build": "grunt uiFramework:build"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/elastic/kibana.git"
  },
  "dependencies": {
    "@elastic/datemath": "2.3.0",
    "@elastic/filesaver": "1.1.2",
    "@elastic/httpolyglot": "0.1.2-elasticpatch1",
    "@elastic/leaflet-draw": "0.2.3",
    "@elastic/leaflet-heat": "0.1.3",
    "@elastic/numeral": "2.2.2",
    "@elastic/test-subj-selector": "0.2.1",
    "@elastic/ui-ace": "0.2.3",
    "@elastic/webpack-directory-name-as-main": "2.0.2",
    "JSONStream": "1.1.1",
    "accept-language-parser": "1.2.0",
    "angular": "1.4.7",
    "angular-bootstrap-colorpicker": "3.0.19",
    "angular-elastic": "2.5.0",
    "angular-route": "1.4.7",
    "angular-sanitize": "1.5.7",
    "angular-sortable-view": "0.0.15",
    "angular-translate": "2.13.1",
    "ansicolors": "0.3.2",
    "autoprefixer": "6.5.4",
    "autoprefixer-loader": "2.0.0",
    "babel-cli": "6.18.0",
    "babel-core": "6.21.0",
    "babel-jest": "20.0.3",
    "babel-loader": "6.2.10",
    "babel-plugin-add-module-exports": "0.2.1",
    "babel-plugin-transform-async-generator-functions": "6.24.1",
    "babel-plugin-transform-class-properties": "6.24.1",
    "babel-plugin-transform-object-rest-spread": "6.23.0",
    "babel-polyfill": "6.20.0",
    "babel-preset-env": "1.4.0",
    "babel-preset-react": "6.22.0",
    "babel-register": "6.18.0",
    "bluebird": "2.9.34",
    "body-parser": "1.12.0",
    "boom": "5.2.0",
    "brace": "0.5.1",
    "bunyan": "1.7.1",
    "check-hash": "1.0.1",
    "color": "1.0.3",
    "commander": "2.8.1",
    "css-loader": "0.28.1",
    "d3": "3.5.6",
    "d3-cloud": "1.2.1",
    "dragula": "3.7.0",
    "elasticsearch": "13.0.1",
    "elasticsearch-browser": "13.0.1",
    "encode-uri-query": "1.0.0",
    "even-better": "7.0.2",
    "expiry-js": "0.1.7",
    "exports-loader": "0.6.2",
    "expose-loader": "0.7.0",
    "extract-text-webpack-plugin": "0.8.2",
    "file-loader": "0.8.4",
    "flot-charts": "0.8.3",
    "font-awesome": "4.4.0",
    "glob": "5.0.13",
    "glob-all": "3.0.1",
    "good-squeeze": "2.1.0",
    "gridster": "0.5.6",
    "h2o2": "5.1.1",
    "handlebars": "4.0.5",
    "hapi": "14.2.0",
    "imports-loader": "0.6.4",
    "inert": "4.0.2",
    "jade": "1.11.0",
    "jade-loader": "0.7.1",
    "joi": "10.4.1",
    "jquery": "2.2.4",
    "js-yaml": "3.4.1",
    "json-loader": "0.5.3",
    "json-stringify-safe": "5.0.1",
    "jstimezonedetect": "1.0.5",
    "leaflet": "0.7.5",
    "less": "2.7.1",
    "less-loader": "2.2.3",
    "lodash": "3.10.1",
    "minimatch": "2.0.10",
    "mkdirp": "0.5.1",
    "moment": "2.13.0",
    "moment-timezone": "0.5.4",
    "ngreact": "0.3.0",
    "no-ui-slider": "1.2.0",
    "node-fetch": "1.3.2",
    "pegjs": "0.9.0",
    "postcss-loader": "1.3.3",
    "prop-types": "15.5.8",
    "proxy-from-env": "1.0.0",
    "pui-react-overlay-trigger": "7.5.4",
    "pui-react-tooltip": "7.5.4",
    "querystring-browser": "1.0.4",
    "raw-loader": "0.5.1",
    "react": "15.4.2",
    "react-ace": "3.7.0",
    "react-addons-test-utils": "15.4.2",
    "react-anything-sortable": "1.6.1",
    "react-color": "2.11.1",
    "react-dom": "15.4.2",
    "react-input-autosize": "1.1.0",
    "react-markdown": "2.4.2",
    "react-redux": "4.4.5",
    "react-router": "2.0.0",
    "react-router-redux": "4.0.4",
    "react-select": "1.0.0-rc.1",
    "react-sortable": "1.1.0",
    "react-toggle": "3.0.1",
    "reactcss": "1.0.7",
    "redux": "3.0.0",
    "redux-thunk": "0.1.0",
    "request": "2.61.0",
    "resize-observer-polyfill": "1.2.1",
    "rimraf": "2.4.3",
    "rison-node": "1.0.0",
    "rjs-repack-loader": "1.0.6",
    "script-loader": "0.6.1",
    "semver": "5.1.0",
    "style-loader": "0.12.3",
    "tar": "2.2.0",
    "tinygradient": "0.3.0",
    "trunc-html": "1.0.2",
    "trunc-text": "1.0.2",
    "ui-select": "0.19.6",
    "url-loader": "0.5.6",
    "uuid": "3.0.1",
    "validate-npm-package-name": "2.2.2",
    "vision": "4.1.0",
    "webpack": "github:elastic/webpack#fix/query-params-for-aliased-loaders",
    "wreck": "6.2.0",
    "yauzl": "2.7.0"
  },
  "devDependencies": {
    "@elastic/eslint-config-kibana": "0.6.1",
    "@elastic/eslint-plugin-kibana-custom": "1.0.3",
    "angular-mocks": "1.4.7",
    "babel-eslint": "7.2.3",
    "backport": "2.2.0",
    "chai": "3.5.0",
    "chance": "1.0.6",
    "cheerio": "0.22.0",
    "chokidar": "1.6.0",
    "chromedriver": "2.36",
    "classnames": "2.2.5",
    "enzyme": "2.7.0",
    "enzyme-to-json": "1.4.5",
    "eslint": "3.19.0",
    "eslint-plugin-babel": "4.1.1",
    "eslint-plugin-import": "2.3.0",
    "eslint-plugin-jest": "20.0.3",
    "eslint-plugin-mocha": "4.9.0",
    "eslint-plugin-react": "7.0.1",
    "event-stream": "3.3.2",
    "expect.js": "0.3.1",
    "faker": "1.1.0",
    "grunt": "1.0.1",
    "grunt-angular-translate": "0.3.0",
    "grunt-aws-s3": "0.14.5",
    "grunt-babel": "6.0.0",
    "grunt-cli": "0.1.13",
    "grunt-contrib-clean": "1.0.0",
    "grunt-contrib-copy": "0.8.1",
    "grunt-esvm": "3.2.11",
    "grunt-karma": "2.0.0",
    "grunt-run": "0.7.0",
    "grunt-simple-mocha": "0.4.0",
    "gulp-sourcemaps": "1.7.3",
    "highlight.js": "9.0.0",
    "history": "2.1.1",
    "html": "1.0.0",
    "html-loader": "0.4.3",
    "husky": "0.8.1",
    "image-diff": "1.6.0",
    "istanbul-instrumenter-loader": "0.1.3",
    "jest": "20.0.4",
    "jest-cli": "20.0.4",
    "jsdom": "9.9.1",
    "karma": "1.2.0",
    "karma-chrome-launcher": "0.2.0",
    "karma-coverage": "0.5.1",
    "karma-firefox-launcher": "0.1.6",
    "karma-ie-launcher": "0.2.0",
    "karma-junit-reporter": "1.2.0",
    "karma-mocha": "0.2.0",
    "karma-safari-launcher": "0.1.1",
    "keymirror": "0.1.1",
    "leadfoot": "1.7.1",
    "license-checker": "5.1.2",
    "load-grunt-config": "0.19.2",
    "makelogs": "4.0.1",
    "marked-text-renderer": "0.1.0",
    "mocha": "3.3.0",
    "mock-fs": "4.2.0",
    "murmurhash3js": "3.0.1",
    "ncp": "2.0.0",
    "nock": "8.0.0",
    "node-sass": "3.8.0",
    "proxyquire": "1.7.10",
    "sass-loader": "4.0.0",
    "simple-git": "1.37.0",
    "sinon": "1.17.2",
    "source-map": "0.5.6",
    "source-map-support": "0.2.10",
    "strip-ansi": "^3.0.1",
    "supertest": "3.0.0",
    "supertest-as-promised": "2.0.2",
    "tree-kill": "1.1.0",
    "webpack-dev-server": "1.14.1",
    "xml2js": "0.4.19",
    "xmlbuilder": "9.0.4"
  },
  "engines": {
    "node": "^6.14.0",
    "npm": "3.10.10"
  }
}

What is the expected behavior?

It should be pulling source-map version 0.5.6

Please mention your node.js, yarn and operating system version.
node.js version 6.14.3
yarn version 1.7.0
OS Amazon Linux 2

[Mehuge]

I am hitting this issue, even though the dependency is for [email protected] it's pulling in version 0.7.3 which requires node 8.

npm i works, and installs [email protected]
yarn doesn't it bails with the error

error [email protected]: The engine "node" is incompatible with this module. Expected version ">= 8".
error Found incompatible module

yarn add [email protected] --exact gives the same error

Back to npm for now I guess.

Version: [email protected]

[arcanis]

One of the packages in your dependency tree is [email protected], which depends on [email protected]. While 0.5.7 would be a valid resolution, 0.7.3 is just as much. In this particular case, it seems Yarn has opted to install the highest available version possible, rather than use another one from the tree.

Note that even if it works on npm, it's a bit by chance. They don't provide much more guarantees than us in this regard, and no behavior is more right or wrong than the other (for example, if we were to do the opposite, I'm pretty sure someone would open an issue an say "why is 0.X using 0.5.7 instead of the highest available version?").

That said, there's an easy workaround using Yarn custom resolutions. Add the following to your package.json:

{
  "resolutions": {
    "gulp-sourcemaps/source-map": "0.5.7"
  }
}

This will cause Yarn to force gulp-sourcemaps to use the exact version you specified, and ignore whatever is its "true" dependency range.

[elrumordelaluz]

thanks @arcanis for the explanations regarding the _no-guarrantees_ solving the _issue_ switching into npm and the possible _workaround_ using yarn.

What happens if in my case I have lot of _dependencies_ depending on source-map but the error doesn't specify which one is failing.

Here is my npm list source-map --depth=0

├── @babel/[email protected]
├── @babel/[email protected]
├── @babel/[email protected]
├── @react-pdf/[email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

[arcanis]

What I did to find out the problem was coming from gulp-sourcemap was:

• run Yarn using --ignore-engines (so it gets past the engine validation)
• checked the yarn.lock file to see which ranges were resolving to 0.7.3, which can be done with ctrl-f version "0.7.3" (that informed me it was 0.X)
• and finally I searched to see which packages were depending on 0.X by doing ctrl-f source-map "0.X". It brought me directly to gulp-sourcemaps.

Probably easier but less instructive and powerful, once you get a tree (still using --ignore-engines), you can also use yarn why to have some idea why some packages are in your dependency tree.