Arcade: Investigate Arcade failures for version 5.0.0-beta.20221.14

[missymessa]

Arcade Validation run: https://dnceng.visualstudio.com/internal/_build/results?buildId=612731&view=results

Runtime

Runtime build with this version of Arcade: https://dev.azure.com/dnceng/internal/_build/results?buildId=612845&view=results

LKG build used for this branch: https://dev.azure.com/dnceng/internal/_build/results?buildId=611275&view=results

Issues

• SourceLink Validation failure. Problem did not occur in LKG. A bunch of 404s from GitHub. Based on discussion down-thread, it sounds like this is a completely normal error to see based on how we're handling the branching and running of this code (branch from GitHub, push to AzDO to run)

ASPNETCore

ASPNETCore build with this version of Arcade: https://dev.azure.com/dnceng/internal/_build/results?buildId=612842&view=logs&j=1b89928a-2219-5ef9-602f-f95beb3da4dc&t=39263dad-2be6-5f40-bde4-1bab0d7931b4&l=852

LKG build used for this branch: https://dev.azure.com/dnceng/internal/_build/results?buildId=612274&view=logs&j=1b89928a-2219-5ef9-602f-f95beb3da4dc&t=39263dad-2be6-5f40-bde4-1bab0d7931b4

Issues

• Warnings are being treated as errors. This was investigated and the mitigation around this is the following: By removing treatWarningsAsErrors from the override properties list, it’s now getting pulled in from Directory.Build.props. Avoid setting this property to true in Directory.Build.props for the impacted project, since it’s not actually able to restore without warnings.

[missymessa]

@alexperovich are the errors in Runtime's SourceLink Validation stage related to work you recently did?

[missymessa]

Currently attempting to narrow down when warnings started being treated as errors.

Need to validate the following versions that we suspect may have the issue against ASPNETCore:

• 5.0.0-beta.20206.4 This version treated the nuget warnings as warnings.
• 5.0.0-beta.20206.9 Winner winner, chicken dinner
• 5.0.0-beta.20208.3 Not a reliable version for repo, as other errors manifested instead.

Validating ASPNETCore with Arcade version 5.0.0-beta.20206.4: https://dev.azure.com/dnceng/internal/_build/results?buildId=613933&view=results

Validating ASPNETCore with Arcade version 5.0.0-beta.20206.9: https://dnceng.visualstudio.com/internal/_build/results?buildId=614111&view=results

Validating ASPNETCore with Arcade version 5.0.0-beta.20208.3: https://dnceng.visualstudio.com/internal/_build/results?buildId=614115&view=results

---

The warnings being treated as errors started with version 5.0.0-beta.20206.9. Here is the diff between this version and the previous version where it did not occur: https://github.com/dotnet/arcade/compare/8b448a6c3177fd6f77b481bb232f1e0c09c00032...6c4dea1ab38ae27cd8bea9716e7807c6e01e0acd

/cc @BrennanConroy @sharwell

[missymessa]

SourceLink was failing on Runtime with at least version 5.0.0-beta.20216.3, maybe further back, but those builds were causing failures during the Build stage due to conflicts with Roslyn.

[JohnTortugo]

/cc @safern @Anipik might be able to help about SourceLink for dotnet/runtime in case you need.

[safern]

It seems like the commit SourceLink is looking for doesn't even exist: https://github.com/dotnet/runtime/commits/06d808e15e7081cc2f2e6d1c9a1605f9359c9423

The commit only exists in internal validation branch: https://dev.azure.com/dnceng/internal/_git/dotnet-runtime/commit/06d808e15e7081cc2f2e6d1c9a1605f9359c9423

[missymessa]

@safern should we be concerned about this issue if we were to promote Arcade?

[safern]

The question here is why hadn't we hit this before on validations? Did we use to create a validation branch in the GH repo as well?

[missymessa]

@safern Since about the end of March, we started running Arcade validation using Runtime. The early validations builds we captured were failing at during the Build stage due to conflicts with Roslyn, so we weren't able to see the errors with SourceLink until recently. And yep, we branch in GitHub.

@riarenas is having an epiphany that this may be completely normal.

[safern]

And yep, we branch in GitHub.

But it seems like the commit is missing in GH. This doesn't seem like an issue with source link or source link validation. This is the default behavior I expect, you build from internal, source link takes the build commit, and maps it to github as that is the public repo. If you added this commit, on any branch in GH (matching sha) it will pass the SourceLink validation.

[riarenas]

We branch from Github, but we only push the branch to AzDO. You're right that this is normal.

[missymessa]

@safern @riarenas and I chatted about this and it sounds like this is expected based on how we're handling... okay, Ricardo commented while I was in the middle of writing this :)

[missymessa]

Warnings being treated as errors was investigated, too. Added a comment to the issue description regarding this situation.

[missymessa]

Closing this issue. Investigations complete.