Pkg.jl: What should version in Project.toml be between releases?

[tkf]

If I checkout (say) Compat.jl with ] develop, the version number returned by Pkg.API.installed()["Compat"] is whatever version field is in Compat.jl/Project.toml. In Pkg2, it had "+" after the version number. Here is a concrete example:

(v0.7) pkg> generate HelloWorld
Generating project HelloWorld:
    HelloWorld/Project.toml
    HelloWorld/src/HelloWorld.jl

shell> cd HelloWorld
/tmp/HelloWorld

(HelloWorld) pkg> develop Compat
  Updating git-repo `https://github.com/JuliaLang/Compat.jl.git`
[ Info: Path `/home/takafumi/.julia/dev/Compat` exists and looks like the correct package, using existing path instead of cloning
 Resolving package versions...
  Updating `Project.toml`
  [34da2185] + Compat v0.68.0 [`~/.julia/dev/Compat`]
  Updating `Manifest.toml`
  [34da2185] + Compat v0.68.0 [`~/.julia/dev/Compat`]
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [de555fa4] + IterativeEigensolvers
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [4607b0f0] + SuiteSparse
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

shell> cd ~/.julia/dev/Compat/src/
/home/takafumi/.julia/dev/Compat/src

shell> sed -i "s/^module Compat/\0\nI_EDITED_IT = 1/" Compat.jl

shell> git add .

shell> git commit -m "I edited Compat.jl"
[master 27a0a75] I edited Compat.jl
 1 file changed, 1 insertion(+)

shell> cd -
/tmp/HelloWorld

julia> Pkg.API.installed()["Compat"]
v"0.68.0"

julia> using Compat
[ Info: Recompiling stale cache file /home/takafumi/.julia/compiled/v0.7/Compat/GSFW.ji for module Compat

julia> Compat.I_EDITED_IT
1

julia> versioninfo()
Julia Version 0.7.0-alpha.40
Commit 3a0c6e20e3 (2018-06-06 03:00 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, haswell)

Of course, each package developer can set version = "$X.$Y.$(Z+1)-dev" (say) right after releasing version = "$X.$Y.$Z". If you are going to implement something like PkgDev.tag (#249), it would be nice to automate this.

[StefanKarpinski]

I'm not so keen on the version = "1.2.3" bit in the project file myself—it's got some significant issues.

[KristofferC]

Seems to work for cargo and npm and IIRC most python package managers. What is different about julia? Also seems better than just guessing that everything that is checked out is a higher version than the latest registered version and guessing 0.0 for unregistered packages.

Please elaborate on the "significanct issues"

[StefanKarpinski]

The main issue is that it's inherently never correct. Most things you commit aren't actually the version labeled and there's no mechanism for making sure that it's even remotely accurate. When do you change the version? Before you tag it? What if that turns out not to actually be the version?

[tkf]

When do you change the version?

The reason why I suggested version = "$X.$Y.$(Z+1)-dev" was to make it explicit. For example, git log --oneline (so the time flow bottom to up) can look like

607b1f3e73 Do some edits
9ed7978682 Do some edits
de96d1820c Start developing 1.2.4         # set version = "1.2.4-DEV" in Project.toml
22590d529d (tag: v1.2.3) Release 1.2.3    # set version = "1.2.3" in Project.toml
a29d0d6709 Do some edits
af97963caf Do some edits

where "Release 1.2.3" and "Start developing 1.2.4" are automatically generated by a single command like ] release.

Of course, you can go back to Pkg2 way and generate the developing version on-the-fly. But I think there are some drawbacks:

1. If you want to stick with semantic versioning, you need to bump the patch version and then append -DEV or something since semver only has pre-release version https://semver.org/#spec-item-9 (unlike, say, Python's post-release versioning https://www.python.org/dev/peps/pep-0386/#the-new-versioning-algorithm).
2. If the package developers are already working on feature-addition after 1.2.3 then 1.2.4 won't be released. It'd be confusing to call it 1.2.4-DEV.
3. If the package developers want to develop two branches (e.g., maintenance only requiring patch bump and backward-incompatible major release), only having patch-bump + -DEV is not ideal.

[StefanKarpinski]

Right. Another approach is to only save the major and minor versions in the project file. I.e. version = "1.2" only instead of version = "1.2.3"; of course then you still need to have a way to figure out the full version number. However, we still have that issue with the current arrangement since we don't know if we're before, exactly at, or after the version number that's given since all can and do happen.

[tkf]

So then the full version number would be stored in the git tag? I guess it is a valid solution but personally it sounds nicer if you have single data source for versions, like only in Project.toml or only in git tags.

Another argument for preferring version bump to -DEV right after the release (and storing the whole version number in Project.toml) is that you can actually record when backward-incompatible or feature addition change happens in the git log of Project.toml. For example, if you are committing change for a feature addition, you can simply update Project.toml in the same (or some nearby) commit. This way, it's easier to make sure that a certain version component gets a bump when it's released.

However, we still have that issue with the current arrangement

Yeah, so why don't you guys decide the meaning of version in Project.toml more concrete? It sounds like this is something that you'd want to clarify before everyone starts using Project.toml.

For example, (although it's probably already apparent but) my suggestion is:

1. If version does not have non-numeric pre-release identifier suffix (e.g., 1.2.3 or 1.2.3-alpha.1), it uniquely specifies the version of given checked-out tree.
2. If version ends with non-numeric pre-release identifier (e.g., 1.2.3-DEV) then it specifies the version of the set of pre-release trees.

Of course, to maintain the uniqueness in item 1 without careful manual maintenance, it is ideal to have Pkg command that imposes the constraint.

[StefanKarpinski]

I think it's somewhat unrealistic to think that people developing packages are actually conscious of when they make incompatible API changes and will remember to bump the version number at the time. And once it's in a commit, it's permanent. So then you have what is effectively misinformation permanently committed in your repository. External tagging can at least be fixed in retrospect.

[tkf]

Hmm... I thought that developers are more conscious about the consequence of change in code when they are writing it. For example, I've seen PR comments from PR writer something like "Don't forget to bump minor version since this PR adds some features!" But I know I'm biased toward remembering semver-compatible examples :)

So then you have what is effectively misinformation permanently committed in your repository. External tagging can at least be fixed in retrospect.

Since Git allows branching, you can branch off from the commit introducing changes requiring major or minor version up and release it. You can then merge the branch back to the master if you want (which is likely the case if you do the major bump).

[KristofferC]

NPM:

If you plan to publish your package, the most important things in your package.json are the name and version fields as they will be required. The name and version together form an identifier that is assumed to be completely unique. Changes to the package should come along with changes to the version. If you don't plan to publish your package, the name and version fields are optional.

Cargo:

Let’s take a closer look at Cargo.toml:

[package]
name = "hello_world"
version = "0.1.0"
authors = ["Your Name <[email protected]>"]

Setup.py

__version__ = "3.2"
# setup.py
from coverage import __version__
setup(
   name = 'coverage',
   version = __version__,
   ...
   )

Gradle (java)

version = '1.0'
jar {
   manifest {
       attributes 'Implementation-Title': 'Gradle Quickstart',
                  'Implementation-Version': version
   }
}

Can we discuss what is fundamentally different with Julia so that we can't store the version in the source code like every single other major package manager?

Right now, with old packages, we cannot do proper version resolution with checked out packages because we have no idea what version they have. Guessing on them being the last registered version is just incredibly wrong (the version of a git-commit changes with unrelated tags to a registry, wat).

For the concrete answer to this issue see e.g. https://internals.rust-lang.org/t/confused-about-cargo-version-property/537/2. You bump the version just before you tag.

[StefanKarpinski]

Ok, ok. We should make sure that the tooling enforces that the version field in a tagged version matches. Having a mismatch there would truly be a disaster. What should the version string be pre/post tagged versions? I.e. when do you change it? How do you know of you are before or after?

[martinholters]

I'd say the commit setting the version should be tagged as that version. In the future, attobot could listen for such a commit instead of the tag/GitHub release.

[KristofferC]

In the future, attobot could listen for such a commit instead of the tag/GitHub release.

Yes, this is the Attobot workflow I imagined.

[StefanKarpinski]

What if that version then doesn’t pass its tests or CI verifications?

[martinholters]

Then that becomes an unregistered version? Even now (i.e. pre-Pkg3), one can tag versions that never make into METADATA, but can still keep the tag.

[tkf]

Can we discuss what is fundamentally different with Julia so that we can't store the version in the source code like every single other major package manager?

@KristofferC My guess is that they are not serious about version number of in-development code although I'm reasonably sure only for setup.py.

But I don't strongly argue that Pkg3 should solve this problem. I can see that most people don't find it necessary and solving it perfectly probably complicates the development workflow.

What if that version then doesn’t pass its tests or CI verifications?

@StefanKarpinski One idea to make the whole release process (automated testing, code review and then registration) atomic is to create a release branch and let Attobot handle that branch. Once the "transaction" succeeds, the release branch can be merged into the master; otherwise, revert the release branch, force-push, and then try releasing again.

Then that becomes an unregistered version?

@martinholters Doesn't it mean that the version string before the release is ill-defined? But making it well-defined probably requires a complicated workflow to make the release process atomic as mentioned above. I'm not sure many developers like it :)

[KristofferC]

It is not very important exactly when the version bump is made. The point is so that you can see that when you instantiate a manifest that tracks some old repo branch that the code you are using is at e.g. v0.5.1 when the latest release is 1.2.0. Instead of showing very old code as 1.2.0+ as Pkg2 does (and resolution of other packages should be better as well).

[tkf]

It's fine that Pkg3 don't care in-development version numbers (although it's nice if that's documented). Though you will loose a chance to handle it in a standardized way: for example, Pkg3 can generate something like 1.2.3-DEV.45 by counting number of commits from the latest release as julia is doing. Python has https://github.com/warner/python-versioneer to provide similar feature to packages.

[StefanKarpinski]

But just looking at the version tag is insufficient since the fact that it’s at 1.2.3 only tells you that the actual version is later than 1.2.2 and earlier than 1.2.4. That is quite useful but I don’t see how you can avoid doing what pkg2 does to figure out the version number anyway.

[StefanKarpinski]

Then that becomes an unregistered version? Even now (i.e. pre-Pkg3), one can tag versions that never make into METADATA, but can still keep the tag.

The fact that the current system leads to registered version sequences with holes like Swiss cheese is not exactly great. In the future, it would be preferable to avoid that. I like @tkf’s branch idea. If the version update process was automated I could see the version right before tagging being 1.2.2+ and right after being 1.2.3+ and only actually being 1.2.3 on the actual release commit.

[KristofferC]

The fact that the current system leads to registered version sequences with holes like Swiss cheese is not exactly great.

Could you elaborate on this? What are the holes? How is it different from what we had in 0.6? If your points is that the same version applies to multiple commits then I think this has already been discussed above.

But just looking at the version tag is insufficient since the fact that it’s at 1.2.3 only tells you that the actual version is later than 1.2.2 and earlier than 1.2.4

That's all resolution care about and it is better than having a version for a given commit continuously changing based on external state.

[StefanKarpinski]

The holes come from people tagging a version locally and then trying to register it, which fails and then they keep trying with new tags and skip registering the old, broken tags. It's not new in 0.7, it's a mess in 0.6 and earlier as well. But we can fix it now by improving the process. All you have to do is look at a few PRs on METADATA to see that it's really common to register multiple, consecutive patch versions at the same time, which doesn't make any sense. Not to pick on any particular package or PR but I found e.g. https://github.com/JuliaLang/METADATA.jl/pull/15087/files after poking around for five seconds. This is entirely the fault of the tooling and the workflow. A version number should not be assigned to a source tree until _after_ it is actually clear that the tree is worthy of the version number:

• it passes its own test suite
• it passes registry requirements
• it is compatible with various versions of its dependencies according to SemVer
• it doesn't break packages that depend on it in ways that don't conform to SemVer
• whatever else we want to check

The only way to make sure this works is to arrange that tagged versions are not permanent until _after_ all that stuff is checked. So I think the workflow needs to be something like this:

1. create a release branch
2. bump version in Project.toml from 1.2.3-DEVto 1.2.3 but only on the branch
3. submit the commit on that branch for verification
4. if any verification step fails, delete the branch and start over from master
5. if all verification steps pass
   a. merge the version branch back into master
   b. register that tree as the actual version 1.2.3

It's possible that we can skip the actual branching since we don't need versions to be actual commits anymore, they are associated with source tree states and if you want to register 1.2.3 as a version, all you need to do is take the current source tree state, modify version in Project.toml and then test that tree and register that tree at the end if everything passes. To do that you don't need an actual branch, you just need a tree in which the version number is accurate.

[simonbyrne]

I've started thinking about the new attobot. For the above process to work, we would need:

1. a way to trigger verification, as we would no longer be able to use GitHub releases as we do now.
   
   
   
   • a web interface, or from within Julia?
   
   
   • how do we authenticate that the user has permissions to do this?
   
   
2. full write permissions to the repository contents in order to create a tag
   
   
   
   • there doesn't seem to be a way to do this from a PR, and GitHub App permissions aren't that fine-grained
   
   

In the first iteration of attobot I intentionally tried to avoid these, because I wanted to keep it simple, but also because I don't trust myself enough to code up a system that is sufficiently robust against injection attacks and whatnot.

Technically, those problems aren't insurmountable, but they will require careful planning and thought. On the other hand, if I was new to Julia and registering my package required giving some random app write access to my repo for what is a rather trivial reason, I might have second thoughts about doing it.

[KristofferC]

Why can't we still use tags?

[simonbyrne]

Do you mean trigger the process from a GitHub Release like we do now? Because that would create a tag that points to a commit with a Project.toml file set to version 1.2.3-DEV.

[KristofferC]

Because that would create a tag that points to a commit with a Project.toml file set to version 1.2.3-DEV

Project files only have x.y.z numbers for their version. Whatever number is there when you create the tag is what is getting tagged. That's how cargo and npm work. Tagging a new version is bumping the version number and tagging a github release.

[simonbyrne]

My interpretation of @StefanKarpinski's comment above is that only the actual tagged version should have version 1.2.3 in the Project.toml, all others should be -DEV or whatnot.

[simonbyrne]

I mean, we could just abandon git tags altogether (which seems to be what rust does: https://github.com/rust-lang/cargo/issues/841), but having them match is kind of nice from a developer perspective.

[KristofferC]

Cargo also hosts the published packages, right?

I want to point out that the version in the Project file is only used when you are checking out that package and then it is used to guide the resolver. It is the registry that says what tree is associated with what version.

Anyway, I don't feel like getting into a long discussion about this.
I just know that the release progress which requires you to branch and create -DEV versions etc will be too convoluted for me to be able to use in packages I just maintain unless it is fully automated.

Npm:

When you make changes, you can update the package using
npm version <update_type>
where is one of the semantic versioning release types, patch, minor, or major.
This command will change the version number in package.json.
Note: this will also add a tag with the updated release number to your git repository if you have linked one to your npm account.
After updating the version number, run npm publish again.

Cargo:

Publishing a new version of an existing crate
In order to release a new version, change the version value specified in your Cargo.toml manifest. Keep in mind the semver rules. Then optionally run cargo package if you want to inspect the *.crate file for the new version before publishing, and run cargo publish to upload the new version.

My thought for Julia was:

Publishing a new version of a package
In order to release a new version, change the version value specified in your Project.toml file. Keep in mind the semver rules. Run PkgDev.publish(pkg) or alternatively, if you have AttoBot installed on GitHub, simply tag a new release with the same name as the version.

[simonbyrne]

I just know that the release progress which requires you to branch and create -DEV versions etc will be too convoluted for me to be able to use in packages I just maintain unless it is fully automated.

Agreed.

It is perhaps worth pointing out that for the other package managers mentioned (npm, cargo, pypi via twine) publishing is done via command line utilities, so their workflows are built around that. Using github releases has made things very simple, and also allowed us to avoid issues like authentication, but it is restrictive.

[tkf]

When I suggested the branch-based workflow to @StefanKarpinski, what I was thinking was to setup attobot such that it watches branches release/v*.*.* (say) and pushing such branch triggers the release process. Another slight variant would be to trigger it via PR from a release/v*.*.* to master; this way, attobot can message the user if the release was successful or not. I don't know how GitHub permission works but I suppose this does not require any write permission?

[KristofferC]

I think the way things work now is ok.