Sdk: Microsoft.NET.Test.Sdk is out of date, not flowing properly via darc

[nguerrera]

5.0 has v16.3

https://github.com/dotnet/sdk/blob/release/5.0.1xx-preview1/eng/Versions.props#L60

https://github.com/dotnet/sdk/blob/bbff20c1e2a293835d0fcef16e55ff14bed7c72f/eng/Versions.props#L60

while 3.1.200 has v16.5

https://github.com/dotnet/toolset/blob/release/3.1.2xx/eng/Versions.props#L63

https://github.com/dotnet/toolset/blob/820400bd23e930ac7816c26fc085c1945977a240/eng/Versions.props#L63

[nguerrera]

Seems this is happening manually:
https://github.com/dotnet/toolset/pull/4238

@nohwnd

[nguerrera]

@dsplaisted @mmitche @wli3

[nguerrera]

Looks like this is pushing to the wrong channel?

Microsoft.NET.Test.Sdk @ 16.5.0
Repository:    https://github.com/microsoft/vstest
Branch:        master
Commit:        fd0fe3df347a081b56c4da89f85a3e1ff463a4e7
Build Number:  20200205-01
Date Produced: 2/4/2020 9:05 PM
Build Link:    https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3444183
BAR Build Id:  41910
Released:      False
Channels:
- .NET Core Tooling Dev
Locations:
- https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json (NugetFeed)

I did notice that I got a Microsoft.NET.Test.Sdk when I took the eng feed to get an arcade.

[nguerrera]

Can we make this push to the appropriate channels and therefore be automatic?

[wli3]

@nohwnd Please set automated insertion to high priory. In the meanwhile, you team is responsible to manually tracking test sdk and templates etc to be in right version of SDK and VS. Our team try to keep up, however, as you can see, we could easily miss a release.

[mmitche]

Looks like this is pushing to the wrong channel?

Microsoft.NET.Test.Sdk @ 16.5.0
Repository:    https://github.com/microsoft/vstest
Branch:        master
Commit:        fd0fe3df347a081b56c4da89f85a3e1ff463a4e7
Build Number:  20200205-01
Date Produced: 2/4/2020 9:05 PM
Build Link:    https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3444183
BAR Build Id:  41910
Released:      False
Channels:
- .NET Core Tooling Dev
Locations:
- https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json (NugetFeed)

I did notice that I got a Microsoft.NET.Test.Sdk when I took the eng feed to get an arcade.

Tooling is for general, non-version specific tools (e.g. symreader). It does not some seem like VSTest would fall into that category.

[nohwnd]

Thanks for the info, I am going to talk with my manager to set this on the top of the priority list, let's hope I will be able to correct this soon.

[nguerrera]

I had to revert my attempt to move this forward as it's causing test failures like:

https://dev.azure.com/dnceng/public/_build/results?buildId=545591&view=ms.vss-test-web.build-test-results-tab&runId=17261272&resultId=100223&paneView=debug

Expected string "
Test run for F:\workspace\_work\1\s\artifacts\tmp\Debug\ItTestsWithTh---8EDCCE30\bin\Debug\netcoreapp3.0\win10-x64\XunitCore.dll(.NETCoreApp,Version=v3.0)
Microsoft (R) Test Execution Command Line Tool Version 16.5.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

F:\workspace\_work\1\s\artifacts\bin\redist\Debug\dotnet\sdk\5.0.100-ci\Microsoft.TestPlatform.targets(32,5):
error MSB4132: The \"Microsoft.TestPlatform.Build.Tasks.VSTestTask\" task returned false but did not log an error.
[F:\workspace\_work\1\s\artifacts\tmp\Debug\ItTestsWithTh---8EDCCE30\XunitCore.csproj]" 
"

to contain "Total tests: 2".


Stack trace
   at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
   at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
   at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
   at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
   at FluentAssertions.Primitives.StringAssertions.Contain(String expected, String because, Object[] becauseArgs)
   at Microsoft.DotNet.Cli.Test.Tests.GivenDotnettestBuildsAndRunsTestfromCsproj.ItTestsWithTheSpecifiedRuntimeOption() 
     in /_/src/Tests/dotnet-test.Tests/GivenDotnetTestBuildsAndRunsTestfromCsproj.cs:line 404

[ViktorHofer]

That's because VSTest pushes to .NET Core Tooling Dev (I set that up):

(756) https://github.com/Microsoft/vstest @ master -> .NET Core Tooling Dev

but the subscription points to .NET Core 5 Dev:

https://github.com/Microsoft/vstest (.NET Core 5 Dev) ==> 'https://github.com/dotnet/sdk' ('master')
  - Id: 3c9dee50-6009-4f5e-3201-08d7a4f3bc87
  - Update Frequency: EveryDay
  - Enabled: True
  - Batchable: False
  - Merge Policies:
    Standard

Updated the subscription to point to the right channel. Bits should be flowing now.

[ViktorHofer]

This is now fixed: https://github.com/dotnet/sdk/pull/10806. Closing.

https://github.com/microsoft/vstest/issues/2349 should still be tackled as the whole BAR update procedure currently happens in devdiv's VSTest's official build pipeline. VSTest should use the common arcade infrastructure.