Arcade: [Publishing] Manifest versioning changes affecting darc-add-build-to-channel

[riarenas]

• [x] This issue is blocking
• [X] This issue is causing unreasonable pain

NuGet.client ran into issues when using the add-build-to-channel command to insert one of their builds:

##[error].packages\microsoft.dotnet.arcade.sdk\5.0.0-beta.20330.3\tools\SdkTasks\PublishArtifactsInManifest.proj(79,5): error : (NETCORE_ENGINEERING_TELEMETRY=Publish) This task is not able to handle legacy manifests.
(Build)

C:\dotnet\arcade [master ≡]> darc add-build-to-channel --id 56547 --channel "VS 16.7" --source-branch master
Build 56547 will be assigned to target channel(s) once this build finishes publishing assets: https://devdiv.visualstudio.com/devdiv/_build/results?buildId=3862000
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
The promotion build finished but the build isn't associated with at least one of the target channels. This is an error scenario. Please contact @dnceng.

I suspect https://github.com/dotnet/arcade/commit/5a99acb9c13bddf0174054c3b04c9cfff3fa8586 includes a change that causes NuGet's manifest to be invalid when tring to publish using the command. This is a scenario that should still work.

As a workaround for now, specifying a source-sha parameter to use a commit before the publishing changes allows the build promotion job to succeed

darc add-build-to-channel --id 56547 --channel "VS 16.7" --source-branch "master" --source-sha "a7ba57554594196351a788410800fc6636cef16c"

@epananth We should find out what needs to happen to unblock this scenario. It's possibly related to https://github.com/dotnet/arcade-services/pull/1261 which had to be reverted?

[riarenas]

Do we perhaps need all repos to use a newer Microsoft.DotNet.Build.Tasks.Feed version now, and this has to be updated manually in NuGet.Client?

[epananth]

This is the change that is causing the break -> https://github.com/dotnet/arcade/blob/2dcced7cdf24fac0db9e0217562d0c1431403e99/src/Microsoft.DotNet.Build.Tasks.Feed/src/PublishArtifactsInManifest.cs#L215

Perhaps we can allow both legacy and latest version to succeed for now? what do you think?

[riarenas]

Perhaps we can allow both legacy and latest version to succeed for now? what do you think?

Is it the right fix in this case? Is it fixed by updating the task version used by NuGet to publish their manifest?

[epananth]

If I understood this correctly, they were using an older task feed version to publish into a newer manifest, so that failed.

Upgrading to newer tasks.feed is the best option. Meanwhile if they want to continue using the older version of tasks.feed and older manifest, we can make sure that does not fail by making this change.

[riarenas]

Then is the conditional not necessary? if it isn't necessary to distinguish between versions there, that would be the best, but it does beg the question on why we put it there in the first place.

[epananth]

I think the condition was here just to show that the are using the old version. I need to look more to find out if we really need to distinguish the versions.

[riarenas]

Upgrading this to blocking. @epananth it looks like we need to allow older versions if possible.

[epananth]

Testing this in Nuget repo by passing an extra param PublishingVersion="2" in PushToAzureDevOpsArtifacts. Will update my findings.

[epananth]

Tested a fix in Nuget -> https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3921877&view=results, it works. Need to get this merged next week.

[epananth]

@riarenas can you please work with @zivkan to get this PR merged? (I'm oof next week)

[zivkan]

The PR's been merged. I don't have permissions to close this issue.

[riarenas]

Thanks @zivkan!

I verified that as of this change, adding build to channel for nuget client via darc should be fixed.

Build: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3926903&view=results

darc add-build-to-channel --id 58584 --channel "general testing" --source-branch master
Build 58584 will be assigned to target channel(s) once this build finishes publishing assets: https://devdiv.visualstudio.com/devdiv/_build/results?buildId=3927630
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Waiting '60' seconds for promotion build to complete.
Build '58584' was successfully added to the target channel(s).
Assigning build '58584' to the following channel(s):
        General Testing

Repository:    https://github.com/NuGet/NuGet.Client
Branch:        dev
Commit:        86ab9578971264f8b5a8b1ff1141ea643fd15bab
Build Number:  5.8.0.6752
Date Produced: 7/27/2020 9:23 PM
Build Link:    https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3926903
BAR Build Id:  58584
Released:      False

Closing as resolved.