Yarn: How to upgrade indirect dependencies?

[chinesedfan]

Do you want to request a feature or report a bug?

Feature.

What is the current behavior?
yarn upgrade ignores indirect dependencies, so users can't upgrade them in yarn.lock. If I missed something, please tell me.

If the current behavior is a bug, please provide the steps to reproduce.

• Suppose a new empty project, run yarn add [email protected]
  
  
  
  • 2 indirect dependencies, is-alphabetical and is-decimal, will be installed and saved in yarn.lock
  
  
  • the latest version of is-alphabetical is 1.0.1 now, if another new version, say 1.0.2 was released(to test, you can release 2 test packages by yourself or modify is-alphabetical to be 1.0.0 in yarn.lock, * I know modifying yarn.lock directly is not a regular operation*)
  
  
• No matter which of following ways, yarn always says All of your dependencies are up to date
  
  
  
  • yarn upgrade is-alphabetical
  
  
  • yarn upgrade-interactive
  
  
  • yarn upgrade-interactive is-alphabetical
  
  

What is the expected behavior?
yarn upgrade also supports indirect dependencies.

Please mention your node.js, yarn and operating system version.
Node 8.9.0
yarn 1.3.2
OSX 10.12.6

[milesj]

@chinesedfan Have you tried yarn upgrade-interactive?

[chinesedfan]

@milesj Yes, it results in the same result and I also updated reproduce steps in issue description.

[rally25rs]

That is because yarn add [email protected] sets your package.json to _exactly_ version 1.0.0 as you requested.

yarn upgrade respects your package.json semver range, and since you specified exactly version 1.0.0, it won't offer to upgrade to other versions.

You could resolve this a couple ways:

• yarn upgrade --latest will ignore semver range and see what is tagged as latest in the registry.
• change package.json to accept a version range like ^1.0.0 then yarn upgrade (you might have to yarn install first to get it to update the lock file for the changed range)
• explicitly specify a version to upgrade, like yarn upgrade [email protected] or yarn upgrade is-alphanumerical@^1.0.0

---

Edit:

Sorry, I just noticed there are different package names. alphanumerical and alphabetical look the same at a glance :)

Right, since there is no upgrade for is-alphanumerical, the dependency tree isn't traversed any deeper to handle its transitive dependencies.

You could try adding the --force flag and see if that makes it the subdependencies. Otherwise I think you are right, there isn't an easy way to do that other than yarn remove is-alphanumerical and yarn add is-alphanumerical

[chinesedfan]

@rally25rs Thanks for your reply! I tested 2 more cases.

• yarn upgrade is-alphabetical --force doesn't work, either.
• yarn upgrade is-alphanumerical will upgrade ALL its subdependencies even if it is already latest.
  
  
  
  • But if I just want to upgrade a specified subdependency, that is still not very convenient.
  
  

[OneCyrus]

yes, this is a major problem with yarn at the moment. and it's already in discussion at #2394

[rally25rs]

duplicate #2394

[AlexWayfer]

Please, re-open: it's not duplicate.

2394 describes duplicating of meck-test-bb package (indirect dependency):

I got two copies of meck-test-bb

This issue describes just ability to upgrade indirect dependency (somehow).

[chinesedfan]

@rally25rs Forgive me to ping.

[AlexWayfer]

@rally25rs, please, you've closed both non-duplicating issues, it's wrong. Give us ability to upgrade indirect dependencies, please!

[rally25rs]

Sorry, there was some confusion over on the other issue. I originally thought 2394 was asking for a way to upgrade a transitive dep using a --deep flag, or something like that, so I had marked this issue as a duplicate of that. Later on after re-reading 2394 I think it was about something different, which is not a duplicate of this like I originally thought.

[alex-thewsey-ibm]

+1 for this feature request. Also an example for anybody dumb like me who needs to upgrade a specific indirect dependency manually in the interim:

Given explicit dependency jsonwebtoken has resolved implicit dependency jws^3.0.0 to vulnerablejws=3.1.4: and you need it to instead resolve to patched 3.1.5:

Delete the jws entry e.g. below from yarn.lock, and re-run yarn. The indirect dependency and any affected packages will be updated, without touching other things (on yarn v1.3 at least)

jws@^3.0.0, jws@^3.1.4:
  version "3.1.4"
  resolved "https://registry.npmjs.org/jws/-/jws-3.1.4.tgz#f9e8b9338e8a847277d6444b1464f61880e050a2"
  dependencies:
    base64url "^2.0.0"
    jwa "^1.1.4"
    safe-buffer "^5.0.1"

Edit: Punctuation

[mkutny]

@alex-thewsey-ibm, thanks for the workaround!

Worked on yarn v1.7.

[Subtletree]

ty, worked Yarn 1.9.2

[joelpurra]

Might help to nudge yarn with selective dependency resolutions, even if it's for a single dependency. Thanks to @remolueoend for the hint!
https://yarnpkg.com/lang/en/docs/selective-version-resolutions/

From the docs:

{
  "name": "project",
  "version": "1.0.0",
  "dependencies": {
    "left-pad": "1.0.0",
    "c": "file:../c-1",
    "d2": "file:../d2-1"
  },
  "resolutions": {
    "d2/left-pad": "1.1.1",
    "c/**/left-pad": "1.1.2"
  }
}

[ai]

We need this feature in Autoprefixer to suggest users how to update caniuse-lite in their yarn.lock https://github.com/postcss/autoprefixer/issues/1184

[richseviora]

Same problem here. I expected yarn upgrade caniuse-lite browserslist to upgrade the sub-dependency. It didn't do that, nor did it give me an error message saying that it can't upgrade it b/c it's not a dependency.

Deleting the relevant lockfile entries and then re-running yarn as @alex-thewsey-ibm suggested fixed the immediate issue for me.

[jrochkind]

It is odd to me that yarn is missing tnis feature. I am new to yarn (and npm), so assumed there must be a way to do this. I'm still not completely sure if there is a non-obvious way to do this that none in this thread know, or if there's really no way to do this.

If there's really no way to upgrade a transitive/indirect dependency in the lockfile without adding it to package.json... I don't understand how yarn users do without it.

[Machiaweliczny]

This is unexpected behaviour IMO - I tried to update lodash recently with yarn upgrade [email protected] and it still haven't updated for any transient dependency.

I was still left with all major(^4.X.X) and patch(~4.17.X) versions pointing to old version.

Only way to fix it is via manual editting of yarn.lock and then maybe running yarn upgrade to consolidate changes. I would expect a little better from such widely used tool.

Is this acknowledged bug or yarn is conservative by default and I'm expected to manually edit yarn.lock or use some flag?

[cheshrkat]

I suspect I have the same security alert to resolve as @Machiaweliczny ;) It would be really useful to override the indirect dependency while we wait on projects to fix their own. Even with highly responsive projects there's a delay waiting for fixes and releases.

[djmitche]

Indeed, this is problematic for security issues in indirect dependencies, and the workaround of editing yarn.lock, while effective, is disappointing and difficult to automate. If this is not the default behavior for some reason, it'd be great to add an option like --include-indirect that will force Yarn to follow the indirect dependencies.

[jrochkind]

I don't think it should need an option, I don't see why yarn upgrade [an indirect dependency] doesn't just update the yarn.lock to latest version of the indirect dependency that is allowed by dependency tree, without any need for additional options. I think right now it's just a no-op?

However, another workaround I found I am happy with is adding resolutions to my package.json. If lodash is an indirect dependency, and I know that I need it to be >= 4.7.13 to avoid a security vulnerability, I can add to my package.json:

  "resolutions": {
    "lodash": ">= 4.17.13"
  }

Then just run yarn install, it will update the yarn.lock to meet that requirement, or complain if it can't because it conflicts with an indirect dependency.

This actually seems to have worked pretty well in my case; I wonder if it's not a "workaround" but the intended solution? It took me a while to discover though. And I don't understand things well enough to be sure this is a universal/correct solution or if there might be any problems in some cases with it. If it is the 'right' solution for upgrading indirect dependencies, it was somewhat hard to find.

[milesj]

Why not yarn install the transitive dep you want to update but don't commit the package.json changes, just the yarn.lock?

[djmitche]

Why not yarn install the transitive dep you want to update but don't commit the package.json changes, just the yarn.lock?

I don't think that will (always?) work, because yarn will happily install multiple versions of the same package, even if a single version would satisfy all of the relevant semver ranges. So this would install the version you want, but not remove the version you don't want.

[milesj]

@djmitche Right. You would need to install the version within the range expected. Not ideal and a bit tedious, but an available stopgap for now.

[kumar303]

@djmitche Indeed, this is problematic for security issues in indirect dependencies, and the workaround of editing yarn.lock, while effective, is disappointing and difficult to automate.

This is another workaround which is slightly more automatable:

yarn remove is-alphanumerical
yarn add is-alphanumerical

Using the example in the PR description, this would remove the top level dep then re-add it which will get all of its latest sub-deps, according to the ranges specified by is-alphanumerical (caret ranges, for example). It will then produce a new lock file.

The side effect is that it will update all sub-deps which is not ideal. For a security issue in sub-dep A I'd want to only update sub-dep A.

The workaround of adding sub-dep A as a direct dep just to fix a security issue is worse because it creates confusion (the package is not used directly) as well as a maintenance burden.

[Cellule]

yarn remove is-alphanumerical
yarn add is-alphanumerical

This seems to be the only reliable way to actually update a dependency. I realized today that we were stuck on a 1.0.0 version of a package that had updated to 1.1.0 a year ago. The package we were using did use ^1.0.0 and all the times we "upgraded" that package it never picked up the new 1.1.0 version of its dependency.
Turns out there was a pretty bad bug that were silently failing in our product that should have been fixed a year ago without me wasting a day investigating why we were having this issue.

I can't believe editing yarn.lock manually, removing then re-adding the package or using selective resolution are the "ways" to update an indirect dependency.

IMO, yarn upgrade should update the indirect dependencies to the latest accepted semver version, that's why we upgrade a package. I mean, if there's was a break in the semver range it would update the indirect dependencies, so it should do it always.

[djmitche]

Would it help to write some kind of external script to do this? I suppose it would just remove all yarn.lock entries for the given package, and then re-run yarn?

[pftg]

There is a simple script to do this: https://gist.github.com/pftg/fa8fe4ca2bb4638fbd19324376487f42

[nnmrts]

Can one of the maintainers please change the label from cat-feature to cat-bug?

[rally25rs]

Can one of the maintainers please change the label from cat-feature to cat-bug?

why? this isn't a bug. It is as designed. yarn upgrade was never intended to be used to upgrade a transitive dependency. The originally opened "issue" is even labeled as a feature request.

Internally upgrade uses yarn outdated to determine what is out of date and what versions to upgrade to. outdated only checks direct dependencies.

I could be wrong, or perhaps it has changed, but I'm pretty sure that npm upgrade at least as of 3 years ago when yarn upgrade was last reworked, also doesn't provide a way to upgrade a transitive dep. (again, that might have changed since over the years, I'm not too up-to-date on npm's changes).

---

Anyone is free to submit a PR to add this functionality. This is an open source project and it is up to the community at large to contribute. This feature request has been open for years but no one has stepped up to implement it and that is why it hasn't been "fixed".

[pftg]

@nnmrts could you check my script https://github.com/yarnpkg/yarn/issues/4986#issuecomment-562719589 will it help you for now?

[nnmrts]

@rally25rs Sorry, I was tired and grumpy, consider my comment as resolved. 😬

[nnmrts]

@nnmrts could you check my script #4986 (comment) will it help you for now?

That script unfortunately didn't work for me, tried it out yesterday. Maybe I did something wrong, but my whole yarn.lock file got "emptied" by the script.

[dacioromero]

I'm not sure how good of a workaround this is, but I ran this script that I wrote:

const fs = require('fs')
const lockfile = require('@yarnpkg/lockfile')
const package = require('./package.json')

const lock = lockfile.parse(fs.readFileSync('yarn.lock', 'utf-8')).object

const allDeps = new Set()

const parseDep = ([name, version]) => {
  allDeps.add(`${name}@${version}`)
}

Object.entries(package.dependencies).forEach(parseDep)
Object.entries(package.devDependencies).forEach(parseDep)

const newLock = Object.fromEntries(Object.entries(lock).filter(([dep]) => allDeps.has(dep)))
const newLockString = lockfile.stringify(newLock)

fs.writeFileSync('yarn.lock', newLockString)

Then ran yarn install and it seems to install the latest verisons of indirect dependencies.

[ayushya]

I was able to resolve deep/indirect dependencies successfully. I wonder when we will get an official support.

https://medium.com/@ayushya/upgrading-javascript-packages-deep-dependencies-using-yarn-8b5983d5fb6b

I have tried to resolve and explained the risks of re-generating the yarn.lock and have suggested what should be done.

Let me know if this works for you guys as well. Or any suggestions to improve the process of upgrading.

[jrochkind]

@ayushya Hm, that does seem to work, genius.

I wonder if yarn would accept a patch in which yarn upgrade to an indirect dependency (or some other command?) just... did that?

[natematykiewicz]

@jrochkind I would have expected a yarn upgrade of an indirect dependency to upgrade it even if it's not my direct dependency. Without that feature, you can be years behind on updates of indirect dependencies.

In my case, I was trying to upgrade fsevents, so that it didn't spew errors when I did a yarn install (https://github.com/fsevents/fsevents/issues/278). fsevents isn't a package I directly use -- it's something that webpack-dev-server uses. But shockingly, I was locked to whatever version existed when webpack-dev-server was first installed in this app.

I had to use this trick to upgrade it, which seemed like a total hack. https://github.com/yarnpkg/yarn/issues/4986#issuecomment-395036563

[FelipeLujan]

the workaround is not working for me, unfortunately, the old deep-dependencies were added again to the yarn.lock file after deleting them. I can see them back in the node_modules folder and yarn.lock file after running yarn install.

maybe the workaround is not compatible with yarn workspaces?

[jrochkind]

@FelipeLujan when the workaround works, the deep-dependencies are still added again to the yarn.lock file -- that is expected -- but with new later versions. But only with new versions if there are new versions released, and if they are allowed by the dependency tree. If some intermediary dependency expresses a restriction which doesn't allow an upgrade, they still can't be upgraded. They are just upgraded to the latest allowed by the restrictions in the tree.

i don't use yarn workspaces though, so can't say if that's relevant.

[ayushya]

@FelipeLujan AFAIK yarn workspaces work in a similar way.

The workaround to delete the package section will only upgrade the package to the latest MINOR/PATCH version.
If you want to upgrade the Package to a newer MAJOR version you have to find the package dependency chain running yarn why package-name-here and upgrade the package at the top of its chain.

CAUTION: Do test your code as upgrading the packages to a newer MAJOR version might bring some breaking changes.

[djmitche]

https://github.com/djmitche/yarn-minify might help with this..

[jrochkind]

@ayushya's workaround is working great for me, manually editing the yarn.lock to remove an indirect dependency, then running yarn install to re-add it at a later version.

To me, this seems like a feature that should be built into yarn though, rather than requiring you to manually edit yarn.lock. It seems like it should be fairly striaghtforward to make a feature that is as if you had manually edited to delete the dependency and run yarn install. I feel this feature really oughta be built into yarn, and am kinda confused as to why it isn't.

[pramoditagi]

I was able to resolve deep/indirect dependencies successfully. I wonder when we will get an official support.

https://medium.com/@ayushya/upgrading-javascript-packages-deep-dependencies-using-yarn-8b5983d5fb6b

I have tried to resolve and explained the risks of re-generating the yarn.lock and have suggested what should be done.

Let me know if this works for you guys as well. Or any suggestions to improve the process of upgrading.

I think that's the same resolution given by @alex-thewsey-ibm. Deleting that particular dependency from yarn.lock helped me.
Anyways, Thanks for this hack☺️

[felipecsl]

Using resolutions in package.json worked for me https://github.com/webpack/webpack-dev-server/issues/2739#issuecomment-695164486

[vfonic]

This should at least return some warning:

yarn add [email protected]
yarn upgrade is-alphabetical

This is what I get instead:

success Saved lockfile.
success Saved 0 new dependencies.

There are zero changes in package.json and yarn.lock even though there's a new is-alphabetical package version available.