Pkg.jl: Deprecate implicit ^ versioning, require the ^ to be written

[oxinabox]

People are constantly getting confused and thinking that:

• the behavour of "1.2.3" is to only allow exactly version "1.2.3" and no other (i.e. the behavour that "=1.2.3" actually has)
• and that "1.2" means 1.2.x" for anyx(i.e. the behavour that"~1.2"actually has, which is also the same as~1.2.0`)

It seems like peoples instinctive understanding of how version specification works is that any digit present is locked and not present is free.
Even though this isn't actually a useful way to decribe versions in SemVer.

When explaining to people the actual behavour, the first thing i find myself saying is that:
"1.2.3" is actually a short-hand for "^1.2.3" and caret versioning means "accept any semver compatible release".
where as equals versioning is accept exactly, and tilde versioning is accept any version with the same major + minor digits/

I think that if it was required to write the caret, people seeing ^1.2.3 would stop and think and learn what it ment, rather than assuming they know (and getting it wrong as above)

[KristofferC]

I don't agree that "people are constantly getting confused", and I also don't think adding a deprecation warning to every single Project would be worth the annoyance.

[StefanKarpinski]

I do regret that we made ^ the default, even though it's the most useful behavior. It's just not intuitive that Foo = "1.2.3" doesn't mean Foo = "=1.2.3" and instead means Foo = "^1.2.3". However, it doesn't seem like a good idea to change this now. The benefit right now is that we can generally just tell people to put their current version in the compat entry and if packages respect semver then that is correct.

[DilumAluthge]

I've definitely seen people get confused on Slack.

I agree with Kristoffer that a deprecation warning seems overkill.

But I would like to make the change for Julia 2.0.

If we do make the change for Julia 2.0 in which we require that the user provide either ^, ~, or =, then instead of deprecation warnings, we can just announce broadly on Discourse, Slack, etc.

[oxinabox]

I agree with Kristoffer that a deprecation warning seems overkill.

Remember that deprecation warnings are opt-in now.

But I would like to make the change for Julia 2.0.

Agreed.
And some time before then we need to deprecate it; so people can change over.
Why would we do this only through an annoucement rather than being told in a deprecation warning?

Though i guess one thing we can do is make it a requirement for AutoMerge.
But that seems more harsh than a opt-in deprecation.

[KristofferC]

Though i guess one thing we can do is make it a requirement for AutoMerge.

Absolutely not, that would be awful. Imagine all the extra work for all manual merges.

To point to some other decent package managers, here is the Cargo docs for compat https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-cratesio.

[oxinabox]

Absolutely not, that would be awful

Agreed

[oxinabox]

To point to some other decent package managers, here is the Cargo docs for compat https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-cratesio.

And here is the npm docs for compat https://docs.npmjs.com/cli/v6/using-npm/semver#ranges
which has it default to equality

Here is the docs from go's old package manager (it seems they have now assended beyond such things as specifying compat)
https://golang.github.io/dep/docs/Gopkg.toml.html#version-rules
which has default of ^
(Though interestly to accomplish this they override the behavour of the semver libary they are using, which defaults to =')

less relevant examples:

for things that don't quite have a ^ specifier.
I got kind of interested this is a cool topic. 😁

Here is ruby's https://guides.rubygems.org/patterns/#semantic-versioning (or better write up: https://devalot.com/articles/2012/04/gem-versions.html)
which doesn't have ~ or ^ but instead has where ~>2.3 is the same as our ^2.3, but ~>2.3.1 is the same as our ~2.3.1.
but anyway if none provided does =.

The Erlang/Elixr package manager Hex, also doesn't have ~ or ^ but does have the ~> also.
AFACT it requires a comparison specifier, though I had to read the source to work that out and I haven't use elixer since one unpleasant experience in 2012 or 2013. So I could be wrong.

Python's pip https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers
doesn't have a ^ or ~ either but does have ~= which is like ruby's ~>
also requires a comparisons specifier.

the .Net package manager NuGet, has ranges written with ( and ] (like we use for documentatation.
But if just a version number is listed then it is treated as out >=.

[oxinabox]

Here is an example of an experienced julia developer getting confused.
I know i have seen others (might edit in here if i find again)
https://github.com/JuliaCollections/DataStructures.jl/pull/710
wether or not the suggestion of getting rid of unspecified versions in 2.0 (and deprecating them now) is the solution, I don't think its reasonable to say people don't often gets confused by this.

[ExpandingMan]

Yes, I'm an experienced Julia developer, and I definitely get easily confused by these. In this example "1.1.0" was allowing "1.3"... which I still find confusing.

That said, this suggestion would seem to be so catastrophically breaking, I'm not sure I agree it would be worth the effort.

[KristofferC]

Did you read the docs? It has examples and everything.

[ExpandingMan]

Yes, I have read them, apparently not recently enough. I'm perfectly willing to admit to feeling pretty stupid for not understanding this better. I completely agree with the original premise of this thread that Package = "1.1" meaning version(Paackage) \ge 1.1 is incredibly confusing (it just looks so wrong), but again I think it might be too late. Literally every single Julia package would have deprecation warnings due to this.

[oxinabox]

Literally every single Julia package would have deprecation warnings due to this.

A silent deprecation warning.
then before 2.0 we can make a bot (like FemtoCleaner) that will go and fix what ever it can for 2.0

[ExpandingMan]

It's probably worth noting that I'm pretty sure the way rust does it corresponds to what @oxinabox and I are saying would be far less confusing.

[DilumAluthge]

Maybe I am misunderstanding, but according to that link, Cargo does exactly what Pkg does:

The string "0.1.12" is a semver version requirement. Since this string does not have any operators in it, it is interpreted the same way as if we had specified "^0.1.12", which is called a caret requirement.

Isn't that what Pkg does? If you don't have an operator, it's interpreted as having a caret?

[ExpandingMan]

I don't know now, I'm so confused because it also says:

An update is allowed if the new version number does not modify the left-most non-zero digit in the major, minor, patch grouping.

Which I take to mean that if you have 1.1.0 it cannot update at all (on the other hand, it does not say "if and only if").

[DilumAluthge]

The left-most non-zero digit in 1.1.0 is the first digit, right?

So an update from 1.1.0 to 1.345.678 is allowed, because it does not modify the first digit?

[ExpandingMan]

:facepalm: I was reading rightmost, my god, I'm really being an idiot today.

I guess the psychological effect of the = sign on me is also overwhelming (even though it does not even mean equality in TOML)

[KristofferC]

It's probably worth noting that I'm pretty sure the way rust does it corresponds to what @oxinabox and I are saying would be far less confusing.

We do the exact same thing as rust do. IIRC, I had to come up with a compat scheme for Pkg, looked around, found the docs for Cargo and pretty much copied it because it made sense and people like Cargo.

What you linked literally has

[dependencies]
time = "0.1.12"

The string "0.1.12" is a semver version requirement. Since this string does not have any operators in it, it is interpreted the same way as if we had specified "^0.1.12", which is called a caret requirement.

[ExpandingMan]

You are right, like I said above, I misinterpreted the rust case as well.

I think that does it then, as much as I agree with the premise of this PR, it really doesn't make any sense to break with the convention of another popular programming language that does things very similarly and simultaneously break tons of Julia packages without a really compelling reason, and some of us having a persistent mental block with this is not that.

[oxinabox]

OTOH: npn (as mentioned above) has opposite convention from Cargo and Pkg's.
npn's default if not specified is = (rather than ^)
npn has many orders of magnitude more users than Cargo.
(Of course question of overlap does come up, and probably highest overlap for julia is not npn or Cargo, but pip and pip requires a specifier (no default), though is generally not as similar to Pkg as npn or Cargo.)
So i don't think matching the convention of another programming language is a compelling argument.

Avoiding change if there is not enough of a reason to change is a compelling argument.
But I suggest that this is a compelling reason to change.
Though if we can think of a better way to make people realize that "1" means "^1" not not =1 nor =1.x.y (the last we don't have but pip, hex, Gem, and npn all do have a way to express. Even though its pretty useless vs ^)