Pkg.jl: Answer to be documented: Is it OK to register a package as 0.x.(y+1) when it is backward-compatible to 0.x.y but has feature additions?

[tkf]

When writing #1416, I realized that Pre-1.0 behavior section doesn't indicate that it is invalid to register a pre-1.0 post-0.1 package without a minor version bump when features are added if it is backward-compatible. SemVer only says that:

How should I deal with revisions in the 0.y.z initial development phase?

The simplest thing to do is start your initial development release at 0.1.0 and then increment the minor version for each subsequent release.

--- https://semver.org/#how-should-i-deal-with-revisions-in-the-0yz-initial-development-phase

But, I've been thinking that I should bump the minor version whenever I introduce new features in pre-1.0 packages. Am I missinterpreting or misremembering something?

If the SemVer rule "feature additions requires minor version bump" only applies to post-1.0 packages in Julia, how about mentioning it explicitly in at the end of Pre-1.0 behavior section?

In particular, it is valid to register a package with version 0.2.4 when it has feature additions compared to 0.2.3 as long as it remains backward compatible with 0.2.0.

I think it has some benefits:

• It makes it more reasonable to write = "0.x" in [compat] as minor version updates become less frequent.

  • As a corollary, it helps advocating or even banning https://github.com/JuliaRegistries/General/issues/3673 registering packages with no upper bounds in [compat] (although this point may be solved soon by the range specification syntax #1410).

• It helps packages to _not_ going 1.0 before solidifying the API. If pre-1.0 packages cannot express backward-compatibility, the only option to do so is to use non-zero major versions. However, a package with ever-changing major versions splits Julia ecosystem in incompatible sub-parts.

• It helps package authors practicing backward-compatibility issues in the wild before going 1.0.

[tkf]

toml-files.md also mentions the pre-1.0 interpretation

https://github.com/JuliaLang/Pkg.jl/blob/44e8977758c625c04400af7995cf79acd6a32b3c/docs/src/toml-files.md#L54-L55

But it also only about "breaking changes."

[DilumAluthge]

Yeah, we should better document our pre-1.0 semver.

Pre-1.0 we don't have major, minor, patch like in normal semver. We only have breaking and non-breaking.

0.3.5 -> 0.4.0 is breaking.

0.3.5 -> 0.3.6 is nonbreaking. New features are welcome as long as they are nonbreaking. If you only have nonbreaking changes, you should use this.

[tkf]

Thanks for clarifying it! I'll make a patch after #1416 is merged (as otherwise I think it may conflict).

[DilumAluthge]

Probably should get one of the Pkg experts to weigh in just to be safe, but I think my explanation is correct.

[KristofferC]

In particular, it is valid to register a package with version 0.2.4 when it has feature additions compared to 0.2.3 as long as it remains backward compatible with 0.2.0.

This is true.

[tpapp]

Strictly speaking,

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

so my understanding is that anything goes. In that sense, the section addressed by #1416 goes beyond SemVer, and should be considered a guideline for the Julia community, that gives kind of an OK default for version resolution in Pkg. Maybe this should be made more explicit?

Incidentally, we should encourage packages to release 1.0 as soon as it makes sense, with the understanding that they can still break stuff and just make it 2.0 etc. Many widely used packages are still languishing in 0.* land.

[tkf]

the section addressed by #1416 goes beyond SemVer

Super strictly speaking, I'd say it's incompatible with SemVer because of the section you quoted. (Personally, I'm thinking that Julia forked the SemVer spec. Not that I'm thinking the bare SemVer is the best way to versioning software. I understand the arguments by Pythonistas that SemVer is too strict and also understand that why Clojure/Go forks think it's too flexible...)

Incidentally, we should encourage packages to release 1.0 as soon as it makes sense, with the understanding that they can still break stuff and just make it 2.0 etc.

IMHO, encouraging major bump too much is risky. I think it would be nice to even mention that updating timescale of major versions should be targeting at least about a year and discouraging 1.0 if they can't foresee stability more than a few months. The emphasis on timescale is somewhat inspired by Numpy's proposed deprecation policy which discusses update frequency of users and deprecation periods. They are not using SemVer but I think the basic idea applies. Of course, I know that not all Julia packages have to be as stable as Numpy.

[KristofferC]

Since no one is tagging 1.0 versions of packages, strictly following SemVer would mean that nothing would be compatible with anything which makes the whole thing pointless.

[DilumAluthge]

Evil idea 😈

We only enable auto-merging for packages with version >= 1.0.0.

People will start tagging 1.0.0 versions of packages pretty quickly

[DilumAluthge]

(I'm only 10% serious)

[KristofferC]

We've been down this road ;) https://github.com/JuliaLang/Pkg.jl/pull/370

[DilumAluthge]

Ahaha fair enough ;P

[tkf]

Ah, so it was the comment https://github.com/JuliaLang/Pkg.jl/pull/370#issuecomment-396961808 that I've got the idea that the answer to my question in the OP could be no. Although now that I'm re-reading it, the comment does not seem to contradict with what has been mentioned in the documentation. I guess I was focused too much on the "breaking releases" perspective.

[tpapp]

Since no one is tagging 1.0 versions of packages

I am not sure what you mean here, quite a few packages are above 1.0 (I did not calculate the proportion though).

nothing would be compatible with anything

Yes, that's why Pkg should make additional assumptions — and I think it is making the right ones.

I think that a blog post or something similar encouraging 1.0 releases now that Julia is stable would be useful. Frankly, Pkg2 (REQUIRE) etc was much more informal about these things, so it is natural that many packages are still in that mindset, but Pkg3 pays much more attention to these nuances so this can change now.

[fredrikekre]

Clarified in https://github.com/JuliaLang/Pkg.jl/pull/1416 ?

[tkf]

I think that a blog post or something similar encouraging 1.0 releases now that Julia is stable would be useful.

Although I do admire all the effort of Julia core team to support backward compatibility based on PkgEval for all the registered packages, I find it unsatisfactory to call a software stable without clearly written deprecation policy. Unfortunately, it seems that the discussion https://github.com/JuliaLang/Juleps/issues/51 is stalled.

Without julia itself having clear deprecation policy, there is no clear reference point for Julia package authors to understand what it means to be 1.0.

Another reason to avoid recommending 1.0 prematurely is that there is no clear definition of public API (I discussed it briefly in https://discourse.julialang.org/t/julias-release-process/28122/8 and https://discourse.julialang.org/t/julias-release-process/28122/14)

[tkf]

(Re-commented here https://discourse.julialang.org/t/how-can-we-encourage-julia-package-developers-to-release-version-1-0-0/29124/5)

[oxinabox]

Without julia itself having clear deprecation policy, there is no clear reference point for Julia package authors to understand what it means to be 1.0.

Julia packages follow semver. Julia follows semver.
SemVer has a pretty explict deprecation policy, that is very clear in the post 1.0 release.

SemVer point 7

Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. ...

SemVer FAQ

How should I handle deprecating functionality?

Deprecating existing functionality is a normal part of software development and is often required to make forward progress. When you deprecate part of your public API, you should do two things: (1) update your documentation to let users know about the change, (2) issue a new minor release with the deprecation in place. Before you completely remove the functionality in a new major release there should be at least one minor release that contains the deprecation so that users can smoothly transition to the new API.

---

The generalization of this to Julia 0.x.y semver extension
is that deprecations additions should be made as increments to y (nonbreaking),
and deprecation removals as increments to x (breaking)

[tkf]

(Isn't it better to keep the discussion in discourse? That's the reason why I quoted my comments there. Also, post-1.0 behavior is irrelevant to this issue.)

But just as a quick reply: I'm talking about the physical timescale which is not mentioned in the SemVer spec IIRC.

[oxinabox]

Aside: Can someone with git permissions move this to JuliaRegistries/General?

[oxinabox]

Also, post-1.0 behavior is irrelevant to this issue

See end of my comment:

The generalization of this to Julia 0.x.y semver extension
is that deprecations additions should be made as increments to y (nonbreaking),
and deprecation removals as increments to x (breaking)

[tkf]

This issue will be closed by #1420.

(Edit: this was in response to https://github.com/JuliaLang/Pkg.jl/issues/1417#issuecomment-534795385)

[tkf]

See end of my comment:

The generalization of this to Julia 0.x.y semver extension
is that deprecations additions should be made as increments to y (nonbreaking),
and deprecation removals as increments to x (breaking)

@oxinabox I discussed deprecation in the context after the discussion in this issue was diverged about recommending 1.0. That's why I thought it might be better to accumulate the discussion in discourse.

[oxinabox]

Oh it is a docs issue. Now I understand.
My bad for only reading the title.

[tkf]

No worries, probably I should have changed the title after the answer was confirmed. I now can see that it is indeed confusing if you come from the discourse thread.