Yarn: CLI - yarn upgrade's behavior is not as expected

[leosco]

I'm running Yarn v0.24.5 on Windows 10 x64 and Node v7.10.0.

Running yarn upgrade doesn't update package.json, only yarn.lock. This can't (or at least shouldn't) be intentional, because first of all, the versioning in package.json is to set the actual version of the package first, and then dictate upgrade constraints, i.e. ^ or ~. Second of all, running yarn upgrade on an individual package changes the package.json so why is the behavior when upgrading all packages different?

[will-stone]

I totally agree. However may I suggest changing the title to something a little less opinionated? It's not "totally flawed" as it does satisfy a function; after all, you're ideally trying to attract the person who wrote this function and they may be less inclined to take a look at an issue that grinds on all their hard work. I'd suggest a tone of "I expect this... , however this is what happens...". Maybe go with "CLI - yarn upgrade's behavior not as expected"?

[dlong500]

FWIW, with npm v5 the npm upgrade (or npm update) command now does update the dependencies in package.json. I believe this is because npm commands now default to adding the --save parameter.

[dlong500]

@leosco I'm not sure I follow your last paragraph implying that version constraints are being ignored. The issue, as far as I can tell is simply that the command doesn't save the updated version identifiers in package.json. It _does_, however, correctly install the latest version of each package matching the semver rules set in package.json.

I personally am a bit torn on what the correct behavior of the command should be. If we are talking about the command always respecting the semantic versioning specified in package.json then updating package.json with the actual installed versions every time the command is run seems of limited value.

In other words, lets say we have a dependency in package.json like so:

"moduleX": "^1.2.0"

If version 1.3.0 of moduleX is released and we run yarn upgrade then v1.3.0 of the package would be installed. But since we aren't actually changing the semantic versioning rules for the package do we really need to update package.json to be the following?

"moduleX": "^1.3.0"

I suppose some people would like the fact that there is an explicit change in the version of a package listed in package.json that would be committed in source control, but it seems to me that the yarn.lock file is where that information should really reside. If you are concerned about committing every change to installed packages, shouldn't yarn.lock be enough? In fact, because package.json doesn't deal with sub-dependencies, it ends up being a rather poor way to keep track of that information and instead just becomes extra noise that clutters source control.

Now if you actually want to change the semver rules for a package (like going from ^1.2.0 to ^2.0.0) then you obviously need a change in package.json, but then we are getting out of the scope of the upgrade command anyway.

It seems to me that there should be a flag for people that want to save every changed version when the command is run, but the default behavior should actually be as it is now.

[will-stone]

I agree, a flag would useful. This would also not confuse users that are used to the current functionality.

[leosco]

@will-stone As long as it's changing the version correctly per your upgrade constraints and reflecting the change in the package.json then it's working as expected, in line with how npm is designed to work. Don't think you need to complicate things with options or flags.

@dlong500 I think you're right, I may have been mistaken about it not following the proper semver constraints, perhaps my versioning in my package.json was incorrect when I first tested it.

About your second point though, this isn't the first time I've heard that, and I really don't agree with the assessment that upgrading a package within version constraints does not need to be reflected in the package.json. Just because your package.json is set to

"dependencies": {
  "dep1": "^1.1.1"
}

doesn't necessarily mean you shouldn't care about updating the package.json to the actual version you're using. It just makes the dependency situation clearer to everyone, and reflects the true state of the world for an application's development.

IMO, It's not enough for a package.json just to say "this app's version of dep1 must greater than or equal to 1.1.1", then some devs have 1.2.0 while others have 1.1.8 and one guy has 1.1.1. That's not really healthy. What the package.json says to me is: "this app's version of dep1 is 1.1.1, but may be upgraded to latest". Then, whenever it is upgraded, it gets updated to "this app's version of dep1 is 1.2.0, but may be upgraded to latest". It also saves the developer the time of manually updating the package.json to reflect new constraints whenever they have to install a new version of a package to access a new feature or deploy a bug fix. Removing the critical responsibility of maintaining functional and up to date dependencies from the potential for human error or forgetfulness is probably the best thing about it. That's my two cents, anyway.

[dlong500]

@leosco In terms of what's "healthy", the scenarios you propose really only get solved by committing the yarn.lock file anyway. Just using package.json to try to "sync" environments between developers has two major problems:
1) It doesn't track sub-dependencies at all, so you could easily end up with multiple developers on the same version of direct dependencies but wildly varying on the versions of sub-dependencies.
2) Because of the semver rules, I don't even know that it's possible to install the actual listed versions of dependencies (without utilizing yarn.lock). An install or upgrade command will always install the latest version of a dependency that satisfies the semver rules. The only way I know of to install specific versions that override semver rules is to individually yarn install [email protected]. So it really doesn't track the information in a helpful way or promote parity between environments IMO. If you really need specific versions of packages and need to carefully curate dependencies then you are free to lock dependencies down to specific versions in package.json, and then you can individually update them when necessary. Or just use the yarn.lock file, as it solves the very issues you are talking about.

In any case, regardless of what the default behavior ends up being, there definitely needs to be a flag to allow users to choose to update or not update the package.json file when running the upgrade command. I know I'd be annoyed at having to deal with changes to that file every time I upgraded dependencies on a bunch of small personal projects where I don't even really need to be tracking anything at all.

[leosco]

@dlong500 Yeah, you're right. The whole point of yarn means the lock file makes syncing deps between devs a seamless process :D so debating effective versioning in package.json isn't a real issue.

My original point still stands though, I think yarn upgrade should still update the package.json, if only for appearances sake. I updated the original issue to remove the part that said semver constraints weren't being met, since I tested it again and I looks like it's working properly according to yarn.lock

[henry74]

This seems to work for me with regards to updating package.json:

yarn upgrade --latest
yarn upgrade <package> --latest
yarn upgrade <package1> <package2> --latest

[dlong500]

@henry74 Those commands will certainly update package.json, but they will also update the versions of packages to the latest available and override any existing semver rules.

If you want to get more specific, you can use the [--caret | --tilde | --exact] flags along with --latest. What seems to be missing to me is the ability to use those flags along with a version specifier (instead of latest) so you could do something like yarn upgrade <package>@2.1.0 --caret. That would give you much more flexibility. Right now it appears, from the CLI, you can only either install the latest version along with setting semver rules or specify an exact version with no capability of setting the semver rules to anything other than exact.

In regards the overall behavior of running upgrade without specifying a version or any flags, I happen to like the behavior as it is now where it will _install_ the highest versions available that match the semver rules but won't update package.json at all. I know that some people would prefer it update package.json with the installed version even when semver rules aren't changing, but to me that seems superfluous and just adds another file that needs to be committed in addition to the yarn.lock file. In any case, I think a flag for updating package.json on any upgrade operation (maybe something like --rewrite) would be a good idea to give people the option.

[henry74]

@dlong500 Ah okay, make sense. I see the nuance differences.

[leosco]

Closing this because it seems people have agreed it's outside of yarn's objective to manage your package.json besides meeting the version constraints themselves. The whole point of yarn.lock is to handle exact versioning, and package.json is treated as legacy.