Cargo: Pre-release version numbers

[SimonSapin]

Should Cargo do something special when a dependency has versions published with pre-release version numbers? Maybe not select them unless requested explicitly with cargo update --precise?

CC https://github.com/whitequark/rust-xdg/pull/9, @whitequark, @alexcrichton

[whitequark]

Specifically, if I put xdg = "2.0" or xdg = ">= 2.0" in Cargo.toml, Cargo selects 2.0.0-pre5, which is in clear contradiction of the semver spec, paragraph 11:

When major, minor, and patch are equal, a pre-release version has lower precedence than a normal version. Example: 1.0.0-alpha < 1.0.0.

[steveklabnik]

Yes, Cargo should follow SemVer here.

I think that this might be the fault of the semver crate itself...

[SimonSapin]

Ok, sorting should be fixed, but that’s separate from the original issue.

[whitequark]

Well, given that * is not a valid dependency spec now, what is left in this issue?

[SimonSapin]

Let me rephrase. Let’s say a package has three versions published: 1.0.0, 1.0.1, and 1.1.0-beta. If I depend on it with version requirement 1.0.0, Cargo will currently pick 1.1.0-beta since it’s the latest. But maybe "pre-release" should signal a version that is published for opt-in testing, but is not ready for general use? In that case, Cargo should default to ignore any pre-release version and pick 1.0.1 instead, unless explicitly requested.

Or, more generally, should we assign meaning (and tool behavior) other than the relative ordering to "pre-release"?

[whitequark]

That is exactly what I am speaking about in https://github.com/rust-lang/cargo/issues/2222#issuecomment-165532916

[SimonSapin]

From the same paragraph that you quoted:

Precedence refers to how versions are compared to each other when ordered.

Precedence is _not_ what I’m talking about.

This part of paragraph 9 is relevant to what I’m talking about, though:

A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version.

Should Cargo consider 1.1.0-beta incompatible with 1.0.0 in the same way that 2.0.0 is incompatible with 1.0.0? (Whereas 1.1.0 _is_ compatible with 1.0.0.)

[whitequark]

As far as I'm aware, Cargo's notion of "compatible with x.y" is just >=x.y <x.(y+1), "compatible with x.y.z" is >=x.y.z <x.y.(z+1), and so on for any amount of parts, which is why precedence counts. The part of paragraph 9 you're talking about is illustrative and follows from the precedence rules.

[whitequark]

I.e. since "compatible with 1.0" is >=1.0 <1.1 and 1.1.0-beta < 1.0, 1.1.0-beta will not be considered "compatible with 1.0".

[SimonSapin]

I.e. since "compatible with 1.0" is >=1.0 <1.1

No. It’s >=1.0 <2.0

[SimonSapin]

I.e. since "compatible with 1.0" is >=1.0 <1.1

No. It’s >=1.0 <2.0

Or rather something like >=1.0 <=1.9999.9999 with an infinity of nines, given how pre-releases sorts.

[whitequark]

Er, sorry, yes, you are correct (4AM here...). But my point still stands.

[alexcrichton]

I agree with @SimonSapin that it seems odd if I say "semver compatible with 1.0.0" that I'll start picking up 1.1.0-beta.1 or whatever new prelease becomes available. I think that Cargo may want to specially treat prerelease versions from that form of compatibility, but I think I've also seen some behavior like this in bundler in the past (@wycats perhaps you could clarify?)

I agree that there also may be an issue with the semver crate which needs to be handled as well, especially if we consider 1.0.0-beta to satisfy a requirement for 1.0.0

[wycats]

Two things that I think are somewhat uncontroversial:

• 1.3.0-beta does not supersede 1.2.0
• 1.3.0-beta-2 supersedes 1.3.0-beta-1

I'd like to suggest canonizing the concept of channels, so that:

• 1.3.0-beta-1 supersedes 1.2.0-beta-6
• 1.4.0-alpha-1 does not supersede 1.3.0-beta-6

The idea is to make the Rust-style release cycle more first class and give people a way, through their Cargo.toml, of subscribing to a particular "release channel".

We could also make subscription explicit through additional metadata:

[dependencies.nix]

version = "1.3.0"
channel = "beta"

This would always select the latest betathat also matches the semver versioning (1.5.0-beta-X would match, but 2.0.0-beta-X would not).

[SimonSapin]

Two things that I think are somewhat uncontroversial:

1.3.0-beta does not supersede 1.2.0
1.3.0-beta-2 supersedes 1.3.0-beta-1

What do you mean by supersedes? “cargo update automatically upgrades form one (in your Cargo.lock file) to the other”? If so I agree.

Note that semver.org only talks about _precedence_, “how versions are compared to each other when ordered”. And 1.3.0-beta does sort after 1.2.0.

I'd like to suggest canonizing the concept of channels, so that:

1.3.0-beta-1 supersedes 1.2.0-beta-6
1.4.0-alpha-1 does not supersede 1.3.0-beta-6

Would there be a list of allowed keywords? semver.org allows arbitrary identifiers for pre-release versions.

[wycats]

What do you mean by supersedes? “cargo update automatically upgrades form one (in your Cargo.lock file) to the other”? If so I agree.

Precisely.

Would there be a list of allowed keywords? semver.org allows arbitrary identifiers for pre-release versions.

Arbitrary identifiers for channels would be allowed, but you could only upgrade across versions on the same "channel".

In practice, it is probably good to stick to a few well-known names like "nightly", "alpha", "beta", "rc", but they would not be interpreted as having any relation to each other across versions.

Just like in Rust, if you subscribe to the "beta" channel, you stick to beta.

[alexcrichton]

One thing we'd need to figure out is what to do when we see a request like:

foo = "1.0.2-beta2"

That's valid by today's rules, but should that _only_ match the package 1.0.2-beta2? Or perhaps 1.0.2-foo would auto-subscribe you to the channel foo?

[jethrogb]

I'm not seeing the behavior that's being talked about in the beginning of the post. In particular, cargo seems to not select prerelease versions at all? Specifying a dependency core_rustc-serialize = "*" (or core_rustc-serialize = "^0.3") results in Cargo choosing 0.3.19, while it should choose the newer 0.3.20-v0.3.19patch1. You can't specify 0.3.20-v0.3.19patch1 in either case by using cargo update --precise. Specifying the exact version string does work. I don't see any code changes linked here that suggest the behavior has changed though.

It seems people in here want cargo to not automatically update to _higher_ prerelease versions, e.g. if the spec is "^0.3.19", it wouldn't update to "0.3.20-alpha". I understand that sentiment, but I think cargo should allow manually updating to that version in this case by using --precise.

[steveklabnik]

cargo seems to not select prerelease versions at all?

In general, it shouldn't select prerelease versions unless you explicitly ask for prerelease versions.

[Frederick888]

Sorry to jump in out of the blue but I've got a quick question: If a package has no stable releases yet, what is the SemVer requirement I should use now to get the latest pre-release?

Currently if I use wildcard, cargo will tell me that there are no matching packages. This problem was originally found in kbknapp/cargo-outdated#75.

[P-E-Meunier]

Another comment on the same topic is, is it expected that cargo install crate installs the pre-release version of crate? I would have expected opt-in testing for this too (would be more consistent with how Cargo behaves for library crates).

[torkleyy]

What's the status here? Can I safely release a 0.7-alpha of my crate?

[Nemo157]

1.3.0-beta-2 supersedes 1.3.0-beta-1

Just a note that anyone considering multiple pre-release versions should read the precedence rules closely (https://semver.org/), while this example is true it’s also the case that 1.3.0-beta-2 supersedes 1.3.0-beta-11. If you want multiple beta releases you should use separate pre-release identifiers like 1.3.0-beta.2. (Similarly if “channels” happen I would hope they’re based on something like literally matching the first pre-release identifier rather than require parsing a concatenated channel+release-in-channel-number from the identifier).

[sjackman]

I have a dependency rust-htslib = { version = "0.26" }. I'd like to locally test the head of the master branch of this repository, version 0.26.2-alpha.0 whose SHA-1 is 0ba0f088e5b5adf032d8d206e23fec509df03a56. This version is in GitHub, but it's not on crates.io. I've used a [patch] block:

[patch.crates-io]
rust-htslib = { git = "https://github.com/rust-bio/rust-htslib.git", rev = "0ba0f088e5b5adf032d8d206e23fec509df03a56"

I've tried the following three cargo update commands.

$ cargo update -p rust-htslib
    Updating crates.io index
warning: Patch `rust-htslib v0.26.2-alpha.0 (https://github.com/rust-bio/rust-htslib.git?rev=0ba0f088e5b5adf032d8d206e23fec509df03a56#0ba0f088)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.

$ cargo update -p rust-htslib --precise=0.26.2-alpha.0
    Updating crates.io index
error: no matching package named `rust-htslib` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
required by package `orbit v0.1.0 (/Users/shaun.jackman/work/orbit)
````

```console
$ cargo update -p rust-htslib --precise=0ba0f088e5b5adf032d8d206e23fec509df03a56
    Updating crates.io index
error: no matching package named `rust-htslib` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
required by package `orbit v0.1.0 (/Users/shaun.jackman/work/orbit)`

I'm working on a crate that depends on rust-htslib = { version = "0.26" } and orbit, and orbit itself also depends on rust-htslib = { version = "0.26" }.

How do I locally test a prerelease version of a dependency?

[woodruffw]

FWIW, a variant of this recently bit me as well: a deeply transitive dependency of a project of mine got bumped from an rc to the next semver-compatible stable version, causing build breakage.

Apart from whether or not this is correct from semver's perspective: it might make sense to have a check on cargo build and/or cargo publish that spits out warnings about unpinned rc (or other pre-tag) dependencies. That at least would nudge people in the right direction :slightly_smiling_face: