Arcade: Feed package version

[chcosta]

How should we version packages in this repo?

In particular, Microsoft.DotNet.Build.Tasks.Feed is currently being published from BuildTools as 2.2.0-prerelease-xxxxx-xx.

W.R.T. packages in arcade:

• Do we want to set the prerelease label to prerelease for everything? Currently it's beta.
• Should we rev the patch version or leave it as 0?

As an example, the test package I produced today out of Arcade is 1.0.0-beta-62706-01. Note that the build number part of the prerelease version is 62706-01 vs 02431-01 from the most recent package published out of buildtools.

I don't imagine that all packages in the repo need to version the same though they'll probably want to share the same prerelease version moniker.

@weshaggard @natemcmaster @markwilkie

[weshaggard]

Based on past experience I do think we are going to want to align these versions with the .NET Core releases. So I suggest we match the same "prerelease" label used for those packages. The actual major/minor versions can be whatever they need to be to allow folks to transition to them. For new packages starting at 1.0.0 seems to be the best.

As for the build numbers I think we need to use the same build numbering scheme we expect to be the standard for all .NET Core repos, as this repo will become the model.

My vote is to use the same versioning model that we use in the core repo's, but I'm of course biased. I think the versioning targets/tasks should be one of the next set of tasks that we pull into this repo.

[chcosta]

BuildTools versioning would make for a fairly simple SDK package. that could be added to the project. It would, however, highlight that Arcade is not yet consuming Arcade.

[chcosta]

@natemcmaster @tmat @jaredpar , is there a dissenting vote against using the core repo's versioning model?

[jaredpar]

The VS teams tend to use -betaX in our packages where X lines up with a particular preview we've shipped to customers. Example we shipped 2.7.0-beta1 which lines up with Dev15.6-preview1.

CC @jasonmalinowski for comment on how important this is to our efforts.

[tmat]

We have never aligned our tools versions with VS versions. I don't think it's necessary. .NET Core (product) has many dependencies that do not version the same, so why would we need that for tool packages?

Roslyn tools (including repo toolset) are currently at 1.0.0-beta2-xxxxx-xx. If we move some of the tools, like RoslynTools.SignTool to Arcade it will need to be renamed or its version will need to be >1.0.0

[jaredpar]

We have never aligned our tools versions with VS versions

Except of course the compiler :)

[natemcmaster]

It looks like this issue was originally about versioning just the Microsoft.DotNet.Build.Tasks.Feed package. I don't care how that one is versioned. But now that it seems we're talking about _all_ packages coming from this repo...

I think there are some important questions go answer clearly first:

• Are all packages released from this repo expected to be the same?
• Are packages going to follow semantic versioning? i.e. a breaking change in API or behavior in a tool should be a major version bump
• Are we interested in giving Semver 2.0.0 a shot? 1.0.0-beta.1234+sha-fc34df1

[tmat]

👎 for including the sha in the package version. I don't think it's necessary. The sha will be included in the NuSpec as metadata at some point. The nuget team is adding that. We will also include the sha in AssemblyInformationalVersion.

[jaredpar]

I'm a fan of putting the SHA into the package version ... and the nuspec, and the assembly and the help strings and really any other place we can shove it.

Having it in the package version as well means I can see what the commit is without having to download the package. That means no need to do a potentially large download, discover the feed the package is on or deal with deleted packages. It's right there in the simplest description of the package.

Version numbers are absolutely meaningless from a distributed build perspective. They are just about as interesting as Guid.NewGuid(). Having the SHA is meaningful though because it assigns a definite answer to the "what is this package" question. Version numbers alone tell us nothing interesting about a package.

[weshaggard]

I know for a fact that we will need to fork and create release branches for Arcade for handling tool fixes during servicing releases. We tried to avoid this in BuildTools but failed miserably so we are going to need to fork these branches for public supported releases of our products, at least .NET Core releases but perhaps other releases like VS as well.

Given that we will have to fork we will need to have some way to produce packages in each branch that doesn't stomp on each other. So we either need to bump the versions each time we release or change the prerelease label. I propose we do the prerelease label (i.e. preview1, preview2) for each fork to help identify some of the changes with our core releases. However we can also update version numbers as well.

Are all packages released from this repo expected to be the same?

I don't think the version numbers need to exactly match but I would like for the prerelease label and build numbers match.

Are packages going to follow semantic versioning? i.e. a breaking change in API or behavior in a tool should be a major version bump

I don't think we will be able to strictly follow this if we align versions with other products but we can definitely attempt to only bump major/minor versions when we make cosponsoring changes in the libraries but I don't think bumping the minor version each time we add a new public API is necessary.

Are we interested in giving Semver 2.0.0 a shot? 1.0.0-beta.1234+sha-fc34df1

I agree with @tmat that we shouldn't be including the sha in the version that just makes it hard to figure out which version is the next version. We will have the sha in many other pieces of metadata.

Another thing to call out is I don't expect any of these packages will ever go stable so they will most likely always have a prerelease label and build number in the version.

[weshaggard]

I agree with @tmat that we shouldn't be including the sha in the version that just makes it hard to figure out which version is the next version. We will have the sha in many other pieces of metadata.

To clarify if we are using semver 2.0 we could include both the build number and the sha and I could live with that. However I do want to point out while we might be able to do that for these tool packages that will be more difficult for our shipping product and we want this repo to be using the same versioning model we will be using in the product.

[tmat]

One thing that I don't understand about the SHA is how is it determined that certain number of digits of the SHA is enough. Lets say fc34df1 is unique at the point the package is built. But it may not be in future. So in future we might end up with two packages that have the same SHA prefix. That's ok versioning wise since the SHA prefix is not used to make it unique. But then finding the actual commit from package version is just a heuristic. Is that considered ok?

[tmat]

@weshaggard Re branches - yes, I'd use betaX or previewX to indicate branch. I'm just saying it doesn't need to be aligned with anything else. It could be unique within the Arcade repo.

[weshaggard]

@weshaggard Re branches - yes, I'd use betaX or previewX to indicate branch. I'm just saying it doesn't need to be aligned with anything else. It could be unique within the Arcade repo.

I agree it could be unique but if we find the reason we end up branching is because of a larger product servicing it would be nice to align the branch name and prerelease label to help things connect to that product a little easier. In fact we could even consider making the prerelease label match the branch name.

[tmat]

In fact we could even consider making the prerelease label match the branch name.

Yes, as long as the branch name isn't release/x.y.z as that's a not valid label. It seems that no two repos agree on what their release branches are called. It's probably reasonable since they ship in many different products with different versioning (VS vs Core vs standalone), so it doesn't make sense to unify on the names.

If we need to patch SignTool for the next VS release but Core is not aligned with that release then we will need a release branch that's different from Core release branch. So while it might seem nice to have the label match I don't think it's gonna work.

[jaredpar]

Lets say fc34df1 is unique at the point the package is built. But it may not be in future. So in future we might end up with two packages that have the same SHA prefix. That's ok versioning wise since the SHA prefix is not used to make it unique. But then finding the actual commit from package version is just a heuristic. Is that considered ok?

I don't see this as a blocking problem. There is a sufficiently log prefix we can choose that will give us a high degree of confidence that it will be a unique identifier. In the case we encounter conflicts we can increase the number of digits going forward.

I agree that means it's a bit of a hueristic. That's fine to me. It's one that's very likely to be correct and useful in a large set of scenarios.

I agree with @tmat that we shouldn't be including the sha in the version that just makes it hard to figure out which version is the next version.

That is an argument for not letting the SHA be the sole identifier of the package. I completely agree with that. Having it be a part of the version though is very sensible. Consider the NerdBank versioning model: it both uses the SHA and has a story for "next version".

We will have the sha in many other pieces of metadata.

Feel like this argument is contradictory. If it's good that we already have it in many other places why is having it in one more bad?

Lacking the SHA in the version string how do I find out what commit represents a package? Download the package? Okay from what feed? What if the package is deleted? What if the package was GC'd by our proposed GC process? If the SHA is there then we don't have to worry about any of these problems.

[tmat]

Lacking the SHA in the version string how do I find out what commit represents a package? Download the package? Okay from what feed? What if the package is deleted? What if the package was GC'd by our proposed GC process? If the SHA is there then we don't have to worry about any of these problems.

Good points.

[weshaggard]

I think I can get on board with including the sha as part of the version. However we will want to do that for all our product prerelease packages as well because we want Arcade to be the model for all our other product repos. Do you guys think that will be possible? I'm not sure what all is necessary in our toolsets to start depending on semver 2.0.

[tmat]

semver 2.0 might be a blocker for product packages. I think we still want Roslyn packages to be consumable by older nuget clients.

[natemcmaster]

On thing that I don't understand about the SHA is how it is determined that certain number of digits of the SHA is enough.

git rev-parse does some fancy math to help us.

https://git-scm.com/docs/git-config

an appropriate value is computed based on the approximate number of packed objects in your repository, which hopefully is enough for abbreviated object names to stay unique for some time.

semver 2.0 might be a blocker for product packages. I think we still want Roslyn packages to be consumable by older nuget clients.

This concern was raised when we talked about it in person with @tannergooding @markwilkie @chcosta @tmat. At the time we discussed it, we said we were interested in giving semver2.0 a shot to tease out where the old NuGet clients are still in use. If possible to upgrade the NuGet client, that's preferred. If that's not possible, we can easily go back to semver 1.0.

[tmat]

git rev-parse does some fancy math to help us.

Yes, it does. But it does not guarantee uniqueness in future, which was my point.

At the time we discussed it, we said we were interested in giving semver2.0 a shot

That was assuming we only do that for our tools, not our product packages.

[natemcmaster]

That was assuming we only do that for our tools, not our product packages.

What are talking about then? I thought this was a discussion about packages coming from dotnet/arcade.

I don't think we will be able to strictly follow this if we align versions with other products

I think we shouldn't attempt to align version numbers with the product. There isn't an clear version to align with. There are too many pieces of the product with many different versions all shipping on different schedules. aspnetcore, .NET Core, VS, MSBuild, NuGet (and more as other teams start using this. Not hard to imagine Azure, IdentityModel, and others wanting to reuse some build tools.) Wasn't the vision of dotnet/arcade shared infrastructure for all of that? IMO let dotnet/arcade version on its own. Obviously, there will be a need to service old versions of build tools, so there needs to be more discussion on how we do this.

Also, if we feel we have to align build tools with a product version, then our build tools are probably too coupled to the product.

My opinion:

• For the first round, let's start everything at 1.0.0.
• let's give semver 2.0 a shot. There are two implications of that
  
  
  
  • Add version metadata. Proposal: +sha-$(CommitHash.Substring(0,10))
  
  
  • if we have build numbers without dashes, use dot-separated syntax. 1.0.0-beta.123 not 1.0.0-beta-123. Semver 2.0 will sort the number segment correctly. If the build numbers have dashes, then we cant use dot-separated syntax but should zero-pad so version sorting is correct
  
  
• for all packages coming out of the arcade repo, let's keep the pre-release label on the package consistent
• the major.minor.patch version can be determined per-package, and use version bumps to indicate servicing/enhancement/breaking changes in the tool.

[weshaggard]

What are talking about then? I thought this was a discussion about packages coming from dotnet/arcade.

The tools that we use to produce the versions in the arcade repo will be the same tools we use to produce the versions in the product repos. Which is why this is coming up in discussion. Now of course we can add options to the version tools to allow support multiple options (semver 1 and 2 as well as allow sha) but the more flexibility we have the less consistency we have across our repos when it comes to versioning.

While we can certainly play around with adding the option for semver 2 with build number and sha to the arcade packages only it seems like that is taking away time from actually producing the tools we need for our shipping products.

Another piece of the puzzle that hasn't really be discussed much yet is how we generate the build numbers that will be used for file version and package version. Each of our repo's do it slightly different today and we are going to have to pick a scheme to share across. I think this is more important then whether or not we start using semver 2 and include the sha.

[chcosta]

I think we need some versioning scheme to get us off the ground. For this issue, in particular, I scoped to the Feed package because, at the moment, it's the only package which has a published corollary and I wanted a non-intrusive solution.

My proposal:

• For the first round, we use the RepoToolset versioning scheme.
  
  
  
  • Feed package versioning example - 2.2.0-prerelease-62706-01
  
  
  • Other Arcade packages versioning example - 1.0.0-prerelease-62706-01
  
  
• Sounds like we're reaching agreement on the long-term versioning strategy, but there will be some work involved there. Once that is decided:
  
  
  
  • Switch Arcade to use Arcade supported versioning scheme, and increment patch (or minor?) version for Arcade packages
  
  

I'm hopeful that we can have an intermediate solution that allows us to move forward and publish packages without pigeon-holing us to a particular scheme (yet).

[chcosta]

@mmitche