yarn upgrade - no update and spurious warning about existing entry in package.json for devDependencies

[jeffrson]

yarn 1.3.0

Do you want to request a feature or report a bug?
Bug (https://github.com/yarnpkg/yarn/issues/1458 is NOT fixed)

What is the current behavior?

If there exists a devDependency when upgrading a package, yarn moans about this because it seems to want to add a new dependency. But doesn't do anything.

If the current behavior is a bug, please provide the steps to reproduce.

yarn init
yarn add @types/node -D
yarn upgrade @types/node

issues
warning "@types/node" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".

What is the expected behavior?

update the devDependency (only) without a warning

Please mention your node.js, yarn and operating system version.
node 9, yarn 1.3.2, Windows 10

[jpickwell]

FYI: you're missing a -D option on the add example.

Node 8.2.1, Yarn 1.3.2 (Homebrew), macOS 10.12

[jeffrson]

Yes, indeed - forgot the "-D" when writing steps to reproduce.

However, as it doesn't change the behaviour I added the "-D" above for correctness.

[jpickwell]

That's interesting that it doesn't change the behavior.

[jeffrson]

Argh - it doesn't change the behaviour I described. If there would be no "-D" there wouldn't be a warning.

Sorry for the confusion.

[jpickwell]

Hehe, ok. No problem.

[krzychukula]

@jpickwell He's not missing -D option. It should not be required in the first place.
I have devDependency node-sass and I want to upgrade it.
yarn upgrade node-sass should be enough. Keeping new version in devDependencies should be a default behaviour not something to require -D to be passed to yarn.
I would call upgrade wanting to move my devDependency do dependencies a bug.

[jpickwell]

@krzychukula, I was referring to the add command. I agree that the -D option shouldn't be needed for the upgrade command. I've never used a -D option on the upgrade command before. This issue is a regression bug, or at least feels like one.

[dawnmist]

This worked properly in version 1.2.x, but is breaking in 1.3.2. It completely breaks the yarn upgrade command - if you have any devDependencies at all (e.g. all the type definition packages for typescript), it is now impossible to use yarn to do any upgrades.

Node 8.9, Linux (LMDE2), Yarn 1.3.2.

[dashmug]

I can confirm that it was working well in version 1.2.x as @dawnmist mentioned. It's broken now with v1.3.2.

[wopian]

On node 9 and yarn 1.3.2. Of the many devDependencies in one project, only the following threw warnings (I guess the rest were already on latest version?)

The upgrade is still successful despite these warnings though, i.e autoprefixer ~7.1.5 is updated to 7.1.6

warning "autoprefixer" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "babel-eslint" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "eslint-plugin-node" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "extract-text-webpack-plugin" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "uglifyjs-webpack-plugin" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "vue-template-compiler" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
warning "webpack-merge" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".

[dawnmist]

This is likely to be where the regression occurred: https://github.com/yarnpkg/yarn/pull/4810

Edit: Yes - yarn upgrade calls the yarn add script's functions, and the add script makes no distinction between whether the user is doing an add directly or has entered there via upgrade. So the rule being enforced for yarn add is mistakenly also being enforced for yarn upgrade - where it will always fail since all dependency types are being updated rather than just one.

[ahmadkelsherif]

I also have this issue
node 6.11.4, Linux, yarn 1.3.2

[Flygenring]

It's not a problem with throwing a warning or calling yarn add, it's a problem with not passing the correct context/flag to the add command to properly update the dependency in the manifest.
True the dependencies gets upgraded correctly and it only throws a warning, but the versions in the manifest will never get updated because of this.

[dandv]

Broken for me too.

git clone [email protected]:dandv/local-iso-dt.git
git checkout f418785b1a1538538f5718c71554f9e443496911
cd local-iso-dt
yarn
yarn upgrade

yarn 1.3.2. The package doesn't have any dependencies. I wanted to upgrade the devDependencies, e.g. babel-cli.

[RomainLanz]

Any news on this? Broken for me as well.

This is a high priority issue....

[dhniels]

Im having this issue as well.

[dvakatsiienko]

yea, me too still

[gorkemkinik]

Also have the issue
node 6.11.1, Linux, yarn 1.3.2

[bwoods43]

I'm also having this same problem - node 9.5.0, yarn 1.3.2. It doesn't seem as if other people at work are having a problem - I'm guessing they are on an older version of yarn.

[cameronmurphy]

Having the issue on macOS, node 8.9.4, yarn 1.5.1

[rally25rs]

this happens because internally upgrade utilizes add by collecting up all the outdated deps and passing them to yarn add, but it does not distinguish between deps and devDeps like add expects.

What we probably need to do is have upgrade break the outdated packages into chunks, deps vs devDeps, and call add twice, once with the -D flag set.

The upgrade of packages is performed correctly, the warning is just an unfortunate incorrect output.

[shepmaster]

@rally25rs does that mean that @dawnmist's comment and @Flygenring's comment were accurate?

[audiolion]

+1 on this, ran into it and was very confused by the behavior (wanted to my my dev dep into regular deps) and it also didnt write anything to my package.json afterwards

[cnshenj]

+1, it's not just an issue of warning message. package.json not being updated is a bug that breaks workflow.

[PhilippMeissner]

Same problem here

Mac 10.13.3
yarn 1.5.1
node 9.10.0

My current workaround is to explicitly upgrade to the very last version that's reported by yarn outdated as in yarn upgrade [email protected].

[rally25rs]

I've mentioned this is several other threads, but yarn upgrade will not update anything in pacakge.json by design, because there is nothing to update. It respects your semver range specified in pacakge.json. yarn upgrade finds the newest version that matches the semver range in package.json and installs it. yarn.lock is updated to the now installed version, and that is the file that matters.

Think of it as: package.json is the acceptable range that you have decided to accept. yarn.lock is the actual version to install. So for example if my package.json specifies phantomjs@^1.0.0 and a version 2.0.0 is released, yarn upgrade will upgrade to the latest 1.x version, because I have told it that I can not accept [email protected].
yarn upgrade --latest would upgrade it to 2.0.0 and so would need to update my package.json with a new acceptable range. The --latest flag basically tells yarn to ignore the specified semver range and act like you were doing a yarn remove phantonjs && yarn add phantomjs.
In both cases yarn.lock will be updated to the version to actually install.

This is documented a bit better in the rfc: https://github.com/yarnpkg/rfcs/blob/master/implemented/0000-upgrade-command-consistency.md

I get that there are like 800 different use cases and desires for how upgrade _could_ work, but this is how it's designed in yarn v1.x (v0.x was different and even more limited).

The warning message is indeed a bug though.

[timhaines]

@rally25rs Thanks Jeff, that was really helpful. Some kind of logging that explained that may be a nice improvement.

[ZebraFlesh]

Perhaps that's the source of confusion then: the breaking change @rally25rs describes when going from 0.x to 1.x was never listed in the 1.0.0 release notes.

[audiolion]

@rally25rs, I don't have an issue with yarn upgrade respecting semver, I expect that, I do have an issue when my devDependency is phantomjs@^1.0.0 and I call yarn upgrade phantomjs --latest and it does not write [email protected] to devDependency and warns me about needing to add it to deps, and passing --dev or -D produces the same

[rally25rs]

@audiolion

when my devDependency is phantomjs@^1.0.0 and I call yarn upgrade phantomjs --latest and it does not write [email protected] to devDependency

🤔 that's interesting. It certainly _should_ write ^2.0.0 to package.json in that case. I can try to reproduce that case here...

[rally25rs]

@audiolion I do get an update to package.json in the case of --latest (example output from a clean directory):

~/Projects/yarn-test 🐒   yarn init -y
yarn init v1.5.1
warning The yes flag has been set. This will automatically answer yes to all questions, which may have security implications.
success Saved package.json
✨  Done in 0.03s.

~/Projects/yarn-test 🐒   yarn add phantomjs@^1.0.0 --dev
yarn add v1.5.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning phantomjs > request > [email protected]: Use uuid module instead
warning phantomjs > request > [email protected]: ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
success Saved 90 new dependencies.
info Direct dependencies
└─ [email protected]
info All dependencies
<package listing omitted>
✨  Done in 6.78s.

~/Projects/yarn-test 🐒   cat package.json
{
  "name": "yarn-test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "devDependencies": {
    "phantomjs": "^1.0.0"
  }
}

~/Projects/yarn-test 🐒   yarn upgrade phantomjs --latest
yarn upgrade v1.5.1
[1/4] 🔍  Resolving packages...
warning [email protected]: Package renamed to phantomjs-prebuilt. Please update 'phantomjs' package references to 'phantomjs-prebuilt'
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Rebuilding all packages...
success Saved lockfile.
success Saved 1 new dependency.
info Direct dependencies
└─ [email protected]
info All dependencies
└─ [email protected]
warning "phantomjs" is already in "devDependencies". Please remove existing entry first before adding it to "dependencies".
✨  Done in 7.96s.

~/Projects/yarn-test 🐒   cat package.json
{
  "name": "yarn-test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "devDependencies": {
    "phantomjs": "^2.1.7"
  }
}

This was on:
Yarn v1.5.1
OSX Sierra
Node 8.10

---

There is one sort-of edge case that I'm aware of, and that is when the currently installed version is already at "latest".

For example is package.json contains phantomjs@^2.0.0 and the actual installed version is 2.1.7, and latest is also 2.1.7, then yarn will check "outdated", which phantomjs will not be (latest is already installed) so will just exit (nothing to do). This means package.json won't change from ^2.0.0 to ^2.1.7 but even that is arguably still correct because the installed version and latest both fit the listed semver range.

However if the currently installed version was 2.1.6 and latest was 2.1.7, then I would expect package.json to be updated to ^2.1.7. Basically it's just the case where yarn outdated shows no changes.

[audiolion]

Ah yep, it is just the logging messages that confused me:

that along with the direct dependencies (making me think regular deps, not dev deps) made me believe it was installing as a regular dep and not a dev dep.

edit: to the case of ^2.0.0 and yarn upgrade --latest and not having it write ^2.1.7, I agree that it is still within semver range and so shouldn't update, if you want it to update you should specifically pin 2.0.0 and then it will update to 2.1.7

edit2: thanks for taking the time to post that example case, I should have done more digging and made the reproducible test case before bringing it to you, my apologies.