Arcade: Create new release pipeline for publishing stable packages

[JohnTortugo]

[jcagme]

What does stable mean in this context?

[JohnTortugo]

This is related to phase 3 of Matt's plan. I understood this would just be a new release pipeline that can identify if packages are stable (based on their version) and push then to a different location other than dotnetfeed. But I'd also like some more context here from @mmitche

[jcagme]

Got it!

[mmitche]

Right, so basically the primary issues are as follows:

• When we have a stable package (e.g. end of release, we stabilize on Microsoft.NETCore.App 3.0.0) the package identity will not change from build to build, even though contents may be different.
• When we have a package identity that doesn't change, we would need to overwrite the feed's current package in order to publish the new one.
• Overwriting is either bad practice in some cases (e.g. dotnetfeed) or not allowed at all (azdo/nuget.org)

So, when we have a stable build, we really need to identify the build as such, and publish the outputs to an isolated feed. I think this feed is effectively temporary. We would then use the idea @jcagme had to propagate the feed info around using darc so that update PRs know about the new feed.

[jcagme]

This is related to https://github.com/dotnet/arcade/issues/1788 since stable assets will end up in a private feed. Also, the release making the publishing could be the same. It should be smart enough to determine if it is pushing stable vs. non-stable packages and with that determine the feed to publish them to.

/fyi: @riarenas

[JohnTortugo]

@mmitche / @tmat - how would be the best way to identify that a build is producing "stable" packages? My plan right now is to add some code here that checks whether "VersionSuffix" is empty or not and emit a ##vso command to set a variable with the outcome of that comparison.

[mmitche]

@JohnTortugo I believe DotNetFinalVersionKind will be set to 'release' for stable packages.

We can feed that into the manifest or whereever we need to influence behavior.

Btw, I'd like some safety checks put into the publish process for feeds that are not isolated (e.g. dotnet-core) to ensure we don't accidentally publish stable packages there if there is a bug, mis-set switch, etc.)

[mmitche]

This is related to #1788 since stable assets will end up in a private feed. Also, the release making the publishing could be the same. It should be smart enough to determine if it is pushing stable vs. non-stable packages and with that determine the feed to publish them to.

/fyi: @riarenas

@jcagme One nit: Stable packages don't need to necessarily be in a private feed, just in an isolated feed. private/public depends on whether the build is public or not.

[JohnTortugo]

@mmitche / @jcagme - is there a plan already for how to tell if the build is private or public?

[mmitche]

@JohnTortugo There's not yet, but it could be as simple as if the branch starts with 'internal/'

[mmitche]

Or, alternatively, you could have BAR figure it out. If commit exists in public repo, then it's public, otherwise private

[markwilkie]

I like the BAR idea better as I'm not super confident in teams following convention.

[JohnTortugo]

Or, alternatively, you could have BAR figure it out. If commit exists in public repo, then it's public, otherwise private

Yeah, I was thinking the same.

[jcagme]

@JohnTortugo I believe DotNetFinalVersionKind will be set to 'release' for stable packages.

We can feed that into the manifest or whereever we need to influence behavior.

Btw, I'd like some safety checks put into the publish process for feeds that are not isolated (e.g. dotnet-core) to ensure we don't accidentally publish stable packages there if there is a bug, mis-set switch, etc.)

This is an example about DotNetFinalVersionKind https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Arcade.Sdk/tools/Version.BeforeCommonTargets.targets#L88.

Or, alternatively, you could have BAR figure it out. If commit exists in public repo, then it's public, otherwise private

We are doing this when we build the asset manifest. We check whether a commit exists in GH or not, we could use that to add something to the manifest.

[mmitche]

Also, we could simply use the channel. Builds are really all internal until they are published. They are only published based on channel. An assignment to an internal-only channel only publishes to internal locations.

[JohnTortugo]

I'd prefer that idea, but I guess we don't have private/public channels concept in Maestro/BAR right now. Right?

[mmitche]

We can add it, but it's implicit in the behavior of the channel release pipeline. e.g. builds assigned to a channel called ".NET Core 3 Internal Servicing" should only publish to internal package feeds.

[JohnTortugo]

I'll write a plan for the issue and send around by e-mail.