Azure-pipelines-tasks: VSTest task fails to execute tests in .NET Core 2.0 test project

[sandersaares]

If I provide a .NET Core 2.0 test project DLL as input to the VSTest task (version 2.0.70), I encounter the following error: 2017-08-21T07:42:45.2798678Z Warning: Unable to load types from the test source 'C:\Agent1\_work\34\s\Tests.NetCore\bin\Release\netcoreapp2.0\Tests.NetCore.dll'. Some or all of the tests in this source may not be discovered.

Indeed, no tests from this assembly are actually executed. The same tests are executed without issues via VS 2017 test explorer.

VS 2017 update 3 and .NET Core SDK 2.0.0 are installed.

[sandersaares]

Possibly related: https://github.com/Microsoft/vstest/issues/596

The workaround from there of manually invoking a different vstest.console.exe and providing /Framework:FrameworkCore10 results in successfully executed tests (but prevents me actually using the VStest task).

[IvanAlekseev]

Same here, for some reason vstest task uses old version of test platform as described here
But even with TPv2 I am unable to get coverage results, here is my log:

2017-08-22T22:31:59.5311961Z ##[section]Starting: VsTest - testAssemblies
2017-08-22T22:31:59.5311961Z ==============================================================================
2017-08-22T22:31:59.5311961Z Task         : Visual Studio Test
2017-08-22T22:31:59.5311961Z Description  : Run tests with Visual Studio test runner
2017-08-22T22:31:59.5311961Z Version      : 2.0.70
2017-08-22T22:31:59.5311961Z Author       : Microsoft Corporation
2017-08-22T22:31:59.5311961Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2017-08-22T22:31:59.5321963Z ==============================================================================
2017-08-22T22:32:01.4448193Z Run the tests locally using vstest.console.exe
2017-08-22T22:32:01.4448193Z ========================================================
2017-08-22T22:32:01.4448193Z Test selector : Test assemblies
2017-08-22T22:32:01.4458193Z Test assemblies : **\██████████████.dll,!**\obj\**
2017-08-22T22:32:01.4458193Z Test filter criteria : null
2017-08-22T22:32:01.4458193Z Search folder : d:\a\1\s
2017-08-22T22:32:01.4458193Z Run settings file : d:\a\1\s
2017-08-22T22:32:01.4458193Z Run in parallel : false
2017-08-22T22:32:01.4458193Z Run in isolation : false
2017-08-22T22:32:01.4458193Z Path to custom adapters : null
2017-08-22T22:32:01.4458193Z Other console options : /Framework:FrameworkCore10
2017-08-22T22:32:01.4458193Z Code coverage enabled : true
2017-08-22T22:32:01.4458193Z vstest.console.exe, specified location : C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\
2017-08-22T22:32:01.4458193Z [command]C:\Windows\System32\Wbem\WMIC.exe datafile where "name='C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Enterprise\\Common7\\IDE\\Extensions\\TestPlatform\\vstest.console.exe'" get Version /Value
2017-08-22T22:32:01.5138201Z 
2017-08-22T22:32:01.5138201Z 
2017-08-22T22:32:01.5138201Z 
2017-08-22T22:32:01.5138201Z 
2017-08-22T22:32:01.5148206Z Version=15.0.0.0
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5148206Z 
2017-08-22T22:32:01.5618235Z ========================================================
2017-08-22T22:32:01.5858253Z ##[warning]Overriding test run parameters is supported only with runsettings file. This option will be ignored.
2017-08-22T22:32:01.6558301Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" d:\a\1\s\██████████████\bin\Release\netcoreapp2.0\██████████████.dll /EnableCodeCoverage /logger:trx "/TestAdapterPath:\"d:\a\1\s\"" /Framework:FrameworkCore10
2017-08-22T22:32:01.7488366Z Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
2017-08-22T22:32:01.7488366Z Copyright (c) Microsoft Corporation.  All rights reserved.
2017-08-22T22:32:01.7488366Z 
2017-08-22T22:32:02.2612970Z Starting test execution, please wait...
2017-08-22T22:32:03.5915507Z M i c r o s o f t   ( R )   C o v e r a g e   C o l l e c t i o n   T o o l   V e r s i o n   1 5 . 0 . 3 0 3 1 9 . 2 
2017-08-22T22:32:03.5915507Z  
2017-08-22T22:32:03.5915507Z  
2017-08-22T22:32:03.5925280Z  C o p y r i g h t   ( c )   M i c r o s o f t   C o r p o r a t i o n .     A l l   r i g h t s   r e s e r v e d . 
2017-08-22T22:32:03.5925280Z  
2017-08-22T22:32:03.5925280Z  
2017-08-22T22:32:03.5925280Z  
2017-08-22T22:32:03.5925280Z  
2017-08-22T22:32:03.7462706Z  2.3852
2017-08-22T22:32:08.2844738Z M i c r o s o f t   ( R )   C o v e r a g e   C o l l e c t i o n   T o o l   V e r s i o n   1 5 . 0 . 3 0 3 1 9 . 2 
2017-08-22T22:32:08.2844738Z  
2017-08-22T22:32:08.2844738Z  
2017-08-22T22:32:08.2844738Z  C o p y r i g h t   ( c )   M i c r o s o f t   C o r p o r a t i o n .     A l l   r i g h t s   r e s e r v e d . 
2017-08-22T22:32:08.2844738Z  
2017-08-22T22:32:08.2844738Z  
2017-08-22T22:32:08.2854744Z  
2017-08-22T22:32:08.2854744Z  
2017-08-22T22:32:08.4942236Z  
2017-08-22T22:32:08.4952237Z Attachments:
2017-08-22T22:32:08.4952237Z   d:\a\1\s\TestResults\e9937387-21c0-4666-a4da-e3f937fd9a54\buildguest_FACTORYVM-AZ191 2017-08-22 22_32_03.coverage
2017-08-22T22:32:08.4962240Z 
2017-08-22T22:32:08.4962240Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2017-08-22T22:32:08.4962240Z Test Run Successful.
2017-08-22T22:32:08.4972226Z Test execution time: 4.7237 Seconds
2017-08-22T22:32:08.5382261Z Results File: d:\a\1\s\TestResults\buildguest_FACTORYVM-AZ191_2017-08-22_22_32_08.trx
2017-08-22T22:32:11.4537548Z ##[section]Async Command Start: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command End: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command Start: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command End: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command Start: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command End: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command Start: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command End: Telemetry
2017-08-22T22:32:11.4537548Z ##[section]Async Command Start: Publish test results
2017-08-22T22:32:12.1438676Z Publishing test results to test run '724'
2017-08-22T22:32:12.1438676Z Test results remaining: 24. Test run id: 724
2017-08-22T22:32:12.3597441Z ##[warning]Skipping attachment as it exceeded the maximum allowed size or not available on disk: d:\a\1\s\TestResults\buildguest_FACTORYVM-AZ191_2017-08-22_22_32_08\In\FACTORYVM-AZ191\buildguest_FACTORYVM-AZ191 2017-08-22 22_32_03.coverage
2017-08-22T22:32:12.6467690Z Published Test Run : https://██████████████/███.Builds/_TestManagement/Runs#runId=724&_a=runCharts
2017-08-22T22:32:12.6467690Z ##[section]Async Command End: Publish test results
2017-08-22T22:32:12.6467690Z ##[section]Finishing: VsTest - testAssemblies

[PBoraMSFT]

@IvanAlekseev, @sandersaares - VSTest task cannot run .NET core tests as it uses the Test platform version 1. To run .NET core tests, we recommend using the .NET core task with the test command.

@IvanAlekseev - Code coverage and other data collection is not supported in test platform v2 yet.

[IvanAlekseev]

@PBoraMSFT it is woking in VS Enterprise locally, and documentation mentions it should work from console https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md#working-with-code-coverage

As you can see in logs posted above I was able to use TPv2 by specifying correct path and have coverage report created, but failed to upload because task is looking for it in different path.

And there is clearly no coverage support for dotnet test now https://github.com/Microsoft/vstest/issues/981

Can you please update VsTest task to use TPv2 and support .net core coverage. This is a real show stopper for us moving to .net core.

[IvanAlekseev]

Also here VS test team says that it is ready

[a0z0ra]

Please reopen this, seems clear to me that the mismatch paths could be the issue.

[nigurr]

Hi,

We are fixing this issue as part of this https://github.com/Microsoft/vsts-agent/pull/1149/files
Will update the thread once fix is gone and new agent is released. thread https://github.com/microsoft/vstest/issues/579#issuecomment-324401462

Just reiterating this point - VSTest task cannot run .NET core tests as it uses the Test platform version 1. To run .NET core tests, we recommend using the .NET core task with the test command.

[sandersaares]

Do I understand you right that when you say "we are fixing this" you mean that VSTest task will be fixed to be able to run .NET Core tests? That is the expected behavior but your later statement there confuses me.

[a0z0ra]

Actually, according to https://github.com/Microsoft/vstest/issues/579#issuecomment-301458975 VSTest can run .NET core tests by /Framework:FrameworkCore10 . I've tried this w/ one test, although it's still buggy and reporting 2 instead of 1 successful test, it is indeed ran by VSTest successfully.

[IvanAlekseev]

@nigurr can we please have more clarification here? Is it going to be fixed for VSTest task or we have to wait for another .net core release to support coverage in dotnet test command?

[PBoraMSFT]

@IvanAlekseev - The VSTest task uses the test platform version 1.
.NET core tests require the test platform version 2, which at this time does not support all the scenarios that are provided by Test platform version 1. The platform team is working on the convergence and once that is done, the task will also be able to consume the converged platform.

If using the vstest.console.exe of the Test platform v2 works for your scenarios, you may use it - but please note that unless the platform convergence happens, our ability to address any issues that may arise here will be limited.

As for this specific issue related to code coverage, we are addressing it. As Nitin indicated earlier, this requires a fix on the agent and will require you to update the agent as the fix becomes available.

[IvanAlekseev]

@PBoraMSFT thanks for you reply. We are currently using hosted agents, so we wait for them to update. Is there any timeline for it and is there any specific place where you notify about updates?

[PBoraMSFT]

We will update on this thread.

[IvanAlekseev]

Today I've found that coverage is now displayed correctly for .net core project

This is what I had to do to make it work

[tiagorosendo]

@IvanAlekseev i've replicated your settings and still have this error, have any idea what that might be?

Params:
2017-10-06T06:06:32.7478934Z Test selector : Test assemblies
2017-10-06T06:06:32.7478934Z Test assemblies : *test.dll,!obj*
2017-10-06T06:06:32.7478934Z Test filter criteria : null
2017-10-06T06:06:32.7478934Z Search folder : d:a1s
2017-10-06T06:06:32.7478934Z Run settings file : d:a1sTest.runsettings
2017-10-06T06:06:32.7478934Z Run in parallel : false
2017-10-06T06:06:32.7478934Z Run in isolation : false
2017-10-06T06:06:32.7478934Z Path to custom adapters : null
2017-10-06T06:06:32.7478934Z Other console options : /Framework:FrameworkCore10
2017-10-06T06:06:32.7478934Z Code coverage enabled : true
2017-10-06T06:06:32.7478934Z vstest.console.exe, specified location : C:Program Files (x86)Microsoft Visual Studio2017EnterpriseCommon7IDEExtensionsTestPlatform

Error:
2017-10-06T06:06:46.7345547Z No test is available in d:a1sUnitTestsbinDebugnetcoreapp2.0xunit.runner.visualstudio.dotnetcore.testadapter.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate and try again.

[IvanAlekseev]

@tiagorosendo Maybe there is a problem with xunit runner. We are using MS Test v2, and it still works fine with settings mentioned above.

[PBoraMSFT]

@tiagorosendo - are you able to run the tests using vstest.console.exe independent of the task?

[tiagorosendo]

@PBoraMSFT u can send me one example ?

[chunliu]

@tiagorosendo it is working fine with xunit. you may want to set the test assemblies to only the assemblies of your test project, then the testadapter.dll will be excluded.

[tiagorosendo]

@chunliu thx, its works! but sonar qube throw one exception when i turn on code coverage =/ i will wait the sonarqube team fix this issue.

[GrimSmiler]

@IvanAlekseev i tried your solution and have the same problem as @tiagorosendo .
We are also using xUnit and we need .net Core 2.0 (which currently not supported).
Thanks for the hint though

[kmvanderpool]

@PBoraMSFT Is the code coverage command available to run on build agents on VSTS? From what I understand it is only available on the Enterprise SKU of Visual Studio so I'm unclear whether it is an option on the build agents that exist on VSTS.

[kmvanderpool]

I tried running the vstest command with the parameters above to enable code coverage and got the error. "System.InvalidOperationException: No test run found." Anyone have an idea of what's causing that error or what might be missing? Do I need to scaffold in some nUnit support to run my xUnit based tests?

I am currently running the version 1 of the .NET Core Task within Visual Studio Online to run the unit tests using xUnit. The projects are currently .NET Core 1.0. Planning on moving to version 2 sometime in Q1 of this year.

---

Starting: VsTest - testRun copy

**************************

Task : Visual Studio Test
Description : Run tests with Visual Studio test runner
Version : 2.3.8
Author : Microsoft Corporation

Help : More Information

In distributed testing flow

Test selector : Test run
Test run Id : '$(test.RunId)'
Search folder : d:a1s
Run settings file : d:a1s
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : /Framework:FrameworkCore10 /logger:console;verbosity="normal"
Code coverage enabled : false
Rerun failed tests: false
vstest.console.exe, specified location : c:Program Files (x86)Microsoft Visual Studio2017EnterpriseCommon7IDEExtensionsTestPlatform
Other console options is not supported when using the multi-agent phase setting. This option will be ignored.
Distributed test execution, number of agents in phase : 1

SystemVssConnection exists true

d:a_tasksVSTest_ef087383-ee5e-42c7-9a53-ab56c98420f92.3.8modulesTestExecutionHost.exe

#

System.InvalidOperationException: No test run found.
at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.StartTestRunScenario() in E:v2.0A7_work15sTaTasksTask.TestExecutionRunTests.cs:line 176
at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.QueueAndWaitForRunToComplete() in E:v2.0A7_work15sTaTasksTask.TestExecutionRunTests.cs:line 91
at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.StartExecution(Dictionary2 arguments) in E:\v2.0\A7\_work\15\s\Ta\Tasks\Task.TestExecution\RunTests.cs:line 25 at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.StartTestRunScenario() in E:\v2.0\A7\_work\15\s\Ta\Tasks\Task.TestExecution\RunTests.cs:line 176 at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.QueueAndWaitForRunToComplete() in E:\v2.0\A7\_work\15\s\Ta\Tasks\Task.TestExecution\RunTests.cs:line 91 at Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.RunTests.StartExecution(Dictionary2 arguments) in E:v2.0A7_work15sTaTasksTask.TestExecutionRunTests.cs:line 25

---

Finishing: VsTest - testRun copy

---

---

[davidmceleney]

Installing 'xunit.runner.visualstudio' package fixed it for me in VS 2017...

[am11]

In terms of PowerShell, VSTS Visual Studio Test task ends up invoking something like:

&"${env:ProgramFiles(x86)}\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" `
  "c:\myproj\bin\x64\Debug\netcoreapp2.0\myproj.Tests.Unit.dll" `
  /EnableCodeCoverage /logger:trx `
  "/TestAdapterPath:`"c:\mycov`"" /platform:x64

With xunit tests, we get lots of:

[xUnit.net 00:00:01.6871767] System.IO.FileNotFoundException : Could not load file or assembly 'Microsoft.VisualStudio.CodeCoverage.Shim, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

Couldn't make it work with /EnableCodeCoverage locally or on VSTS. Without that option, it just works 😞

[alexjustus]

Huge props to @IvanAlekseev for the work-around. We've got code coverage up and running using these settings.

Here's the exact strings required (for easy copy/paste-ability) for anyone that needs them:

Path to vstest.console.exe:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform

Other console options:
/Framework:FrameworkCore10 /logger:console;verbosity="normal"

[eriawan]

any plan to support .NETCore 2.0 unit tests?

Because if I run vstest.console /? I get these info below:

--Framework|/Framework:<Framework Version> Target .Net Framework version to be used for test execution. Valid values are ".NETFramework,Version=v4.5.1", ".NETCoreApp,Version=v1.0" etc. Other supported values are Framework35, Framework40, Framework45 and FrameworkCore10.

And it says it only supports FrameworkCore10.
Will there be FrameworkCore20 or related TFM support for .NET Core 2.0 unit tests available build-in the vstest.console.exe?

cc @PBoraMSFT

[cjfarrelly]

@eriawan for .NETCore 2.0 I use /Framework:".NETCoreApp,Version=v2.0" which is picking up the tests and running them locally but having issues on VSTS hosted agent.

[chamikasandamal]

/Framework:".NETCoreApp,Version=v2.0" fixed the issue for me with .NETCore 2.0

[daliusd]

I actually get VSTS to run tests (.net core 2.0) but it fails with following error:

2018-06-01T11:06:36.1777961Z Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Failed to initialize client proxy: could not connect to test process.
2018-06-01T11:06:36.1779658Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
2018-06-01T11:06:36.1780886Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)

Any ideas what is causing this?

[bryanmacfarlane]

@daliusd - you may want to open a new issue

[daliusd]

@bryanmacfarlane At the moment I solved this problem by using "dotnet test". I might consider opening new issue in the future if need will arise. Still I can answer all the question if someone from VSTS team is interested.

[kiran-mahadev]

Hi @daliusd - have you opened another ticket for the issue you describe above? If yes - please help me with the link. I have run into the same issue.

[daliusd]

I have not done anything on that. Feel free to do that.

[kiran-mahadev]

@daliusd - I've used a dotnet test step to get the tests executed. Have you done anything to get the code coverage working?

[daliusd]

@kiran-mahadev As I have said - nothing. Feel free to open new defect as suggested by @bryanmacfarlane

[andytaw]

I had the same problem.
Fixed by using version 15.0.27924.0 of vstest.console.exe.
I found the exe in C:Program Files (x86)Microsoft Visual Studio2017EnterpriseCommon7IDECommonExtensionsMicrosoftTestWindow.

[joshbouganim]

I can't get the code coverage working for me. All I see is in that tab is "Download Code Coverage results"

here is my yaml:

- task: VSTest@2
  displayName: 'VsTest - testAssemblies'
  inputs:
    testAssemblyVer2: |
     **\$(BuildConfiguration)\*test*.dll
     **\$(BuildConfiguration)\**\*test*.dll
     !**\*Microsoft.VisualStudio.TestPlatform*
     !**\obj\**

    vstestLocationMethod: location

    vstestLocation: 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform '

    codeCoverageEnabled: true

    otherConsoleOptions: '/platform:x64 /Framework:FrameworkCore10 /logger:console;verbosity="normal" '

    platform: '$(BuildPlatform)'

    configuration: '$(BuildConfiguration)'

i also tried /Framework:".NETCoreApp,Version=v2.1" because thats what my app is using

and here is the log for the vstest step:

2018-09-18T14:05:08.2681957Z ##[section]Starting: VsTest - testAssemblies
2018-09-18T14:05:08.2692928Z ==============================================================================
2018-09-18T14:05:08.2693201Z Task         : Visual Studio Test
2018-09-18T14:05:08.2693676Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2018-09-18T14:05:08.2694122Z Version      : 2.139.9
2018-09-18T14:05:08.2694346Z Author       : Microsoft Corporation
2018-09-18T14:05:08.2694607Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-09-18T14:05:08.2694907Z ==============================================================================
2018-09-18T14:05:09.0790545Z SystemVssConnection exists true
2018-09-18T14:05:09.1192328Z SystemVssConnection exists true
2018-09-18T14:05:09.1957968Z Running tests using vstest.console.exe runner.
2018-09-18T14:05:09.1958250Z ======================================================
2018-09-18T14:05:09.1959309Z Test selector : Test assemblies
2018-09-18T14:05:09.1959665Z Test filter criteria : null
2018-09-18T14:05:09.1959950Z Search folder : D:\a\1\s
2018-09-18T14:05:09.1961799Z vstest.console.exe, specified location : C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform
2018-09-18T14:05:09.1962949Z Run settings file : undefined
2018-09-18T14:05:09.1963733Z Run in parallel : false
2018-09-18T14:05:09.1964035Z Run in isolation : false
2018-09-18T14:05:09.1964332Z Path to custom adapters : null
2018-09-18T14:05:09.1965029Z Other console options : /platform:x64 /Framework:FrameworkCore10 /logger:console;verbosity="normal"
2018-09-18T14:05:09.1965361Z Code coverage enabled : true
2018-09-18T14:05:09.1965547Z Diagnostics enabled : false
2018-09-18T14:05:09.1966819Z SystemVssConnection exists true
2018-09-18T14:05:09.1969140Z Run the tests locally using vstest.console.exe
2018-09-18T14:05:09.1969395Z ========================================================
2018-09-18T14:05:09.1969723Z Test selector : Test assemblies
2018-09-18T14:05:09.1970094Z Test assemblies : **\release\*test*.dll,**\release\**\*test*.dll,!**\*Microsoft.VisualStudio.TestPlatform*,!**\obj\**
2018-09-18T14:05:09.1970440Z Test filter criteria : null
2018-09-18T14:05:09.1970713Z Search folder : D:\a\1\s
2018-09-18T14:05:09.1971064Z Run settings file : D:\a\1\s
2018-09-18T14:05:09.1971432Z Run in parallel : false
2018-09-18T14:05:09.1971706Z Run in isolation : false
2018-09-18T14:05:09.1973756Z Path to custom adapters : null
2018-09-18T14:05:09.1974101Z Other console options : /platform:x64 /Framework:FrameworkCore10 /logger:console;verbosity="normal"
2018-09-18T14:05:09.1974429Z Code coverage enabled : true
2018-09-18T14:05:09.1974602Z Diagnostics enabled : false
2018-09-18T14:05:09.1975182Z Rerun failed tests: false
2018-09-18T14:05:09.1975660Z vstest.console.exe, specified location : C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform
2018-09-18T14:05:10.3261280Z ========================================================
2018-09-18T14:05:10.4059260Z ======================================================
2018-09-18T14:05:10.5037701Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" D:\a\1\s\Tests\bin\Release\netcoreapp2.1\Tests.dll /EnableCodeCoverage /logger:trx "/TestAdapterPath:\"D:\a\1\s\"" /platform:x64 /Framework:FrameworkCore10 /logger:console;verbosity=normal
2018-09-18T14:05:11.1305192Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-18T14:05:11.1305812Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-18T14:05:11.1306115Z 
2018-09-18T14:05:14.7480613Z Starting test execution, please wait...
2018-09-18T14:05:15.3091503Z Test run will use DLL(s) built for framework .NETCoreApp,Version=v1.0 and platform X64. Following DLL(s) do not match framework/platform settings.
2018-09-18T14:05:15.3093205Z Tests.dll is built for Framework 2.1 and Platform AnyCPU.
2018-09-18T14:05:15.3093541Z Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
2018-09-18T14:05:15.3093723Z 
2018-09-18T14:05:24.4661106Z M i c r o s o f t   ( R )   C o v e r a g e   C o l l e c t i o n   T o o l   V e r s i o n   1 5 . 0 . 3 0 3 1 9 . 0 
2018-09-18T14:05:24.4662057Z  
2018-09-18T14:05:24.4662257Z  
2018-09-18T14:05:24.4663086Z  C o p y r i g h t   ( c )   M i c r o s o f t   C o r p o r a t i o n .     A l l   r i g h t s   r e s e r v e d . 
2018-09-18T14:05:24.4663841Z  
2018-09-18T14:05:24.4664028Z  
2018-09-18T14:05:24.4664199Z  
2018-09-18T14:05:24.4664385Z  
2018-09-18T14:05:26.0101311Z  2.5304
2018-09-18T14:05:31.1132305Z M i c r o s o f t   ( R )   C o v e r a g e   C o l l e c t i o n   T o o l   V e r s i o n   1 5 . 0 . 3 0 3 1 9 . 0 
2018-09-18T14:05:31.1134394Z  
2018-09-18T14:05:31.1135615Z  
2018-09-18T14:05:31.1136964Z  C o p y r i g h t   ( c )   M i c r o s o f t   C o r p o r a t i o n .     A l l   r i g h t s   r e s e r v e d . 
2018-09-18T14:05:31.1138522Z  
2018-09-18T14:05:31.1139465Z  
2018-09-18T14:05:31.1139999Z  
2018-09-18T14:05:31.1140267Z  
2018-09-18T14:05:32.3604926Z  Passed   ProcessFilesDoesNotThrow
2018-09-18T14:05:32.6319292Z Results File: D:\a\1\s\TestResults\VssAdministrator_factoryvm-az379_2018-09-18_14_05_32.trx
2018-09-18T14:05:32.6340879Z 
2018-09-18T14:05:32.6343928Z Attachments:
2018-09-18T14:05:32.6347247Z   D:\a\1\s\TestResults\b9f7c429-b0f0-4980-a596-81b3b84257dc\VssAdministrator_factoryvm-az379 2018-09-18 14_05_23.coverage
2018-09-18T14:05:32.6348425Z 
2018-09-18T14:05:32.6349458Z Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
2018-09-18T14:05:32.6350511Z Test Run Successful.
2018-09-18T14:05:32.6356135Z Test execution time: 6.3519 Seconds
2018-09-18T14:05:32.9244177Z ##[section]Async Command Start: Publish test results
2018-09-18T14:05:32.9768520Z Publishing test results to test run '11'
2018-09-18T14:05:32.9769345Z Test results remaining: 1. Test run id: 11
2018-09-18T14:05:33.4229435Z Published Test Run : https://dev.azure.com/xx/xx/_TestManagement/Runs#runId=11&_a=runCharts
2018-09-18T14:05:33.4230080Z ##[section]Async Command End: Publish test results
2018-09-18T14:05:33.4231678Z ##[section]Finishing: VsTest - testAssemblies

[Antebios]

here is my yaml:

```

• task: VSTest@2
  displayName: 'VsTest - testAssemblies'
  inputs:
  testAssemblyVer2: |
  $(BuildConfiguration)*test.dll
  *$(BuildConfiguration)**test.dll
  !*Microsoft.VisualStudio.TestPlatform*
  !*obj*

This is the piece I was missing to get rid of my error message of "Unable to find tests". My unit tests were running, but I was trying to figure out how to filter out the TestPlatform dlls. That Pipe for Multiline entry was the trick! But, to answer your question: Nothing is displayed under the "Code Coverage" tab except the link. If you want to see the display, go to the "Summary" tab and then click on the expansion arrow for "Code coverage succeeded" and you will see the progress bar for code coverage.

[joshbouganim]

@Antebios happy to help but following your instructions, this is all i see:

and just for reference, this is my tests tab:

and code coverage tab:

100% of my code is definitely not covered. i'm looking for a more detailed view like in @IvanAlekseev's screenshot that shows blocks and lines covered.

[Antebios]

@joshbouganim - I'm sorry to say but that screenshot that @IvanAlekseev has is the "old" summary page.

[joshbouganim]

so the new code coverage page shows less and is worse? i find that hard to believe. is vstest the problem? .net core?

[nigurr]

@acesiddhu please look into this

[acesiddhu]

@joshbouganim the old UI used to show Block level data as well. we have removed that because we feel user generally needs and interpret Line level data. Dealing with Block is what we think is hard (specially for large code base). Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.

[Huffman17]

Currently my test framework is using .NET Core 2.1 and I do specify this in my runsettings. I am experiencing this same issue in Azure Dev Ops. Locally all I need is my Tests.dll in order to use vstest.console.exe or even dotnet test and it works. However, in Azure Dev Ops, *I build the project, copy just the bin folder for my Test project (We have a data, core & test project in one solution), then publish build artifact to a container. Using this container stored with our runsettings and bin folder/items, I run vstest and the error during test execution is * "_Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further_." How hasnt this been fixed? I do have Microsoft.NET.Test.Sdk as a nuget reference for the project.

Edit: My original comment, I was specifying /Framework: Framework10 Because I read that was a fix where I should have /Framework:.NETCoreApp,Version=v2.1. Now I am still getting the error mentioned here.

[Antebios]

@Huffman17 Did you specify the framework version in the "otherConsoleOptions" arguments?

- task: VSTest@2
  displayName: 'Run Unit Tests'
  inputs:
    testAssemblyVer2: |
     **\$(BuildConfiguration)\*test*.dll
     **\$(BuildConfiguration)\**\*test*.dll
     !**\*Microsoft.VisualStudio.TestPlatform*
     !**\obj\**
    vstestLocationMethod: 'location'
    vstestLocation: 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\'
    codeCoverageEnabled: True
    otherConsoleOptions: '/platform:x64 /Framework:.NETCoreApp,Version=v2.1 /logger:console;verbosity="normal" '
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

[Huffman17]

@Antebios Yes, I have specified it that way, I was specifying framework in the wrong format I guess, oddly enough. I get the error still _"Unable to find tests for D:ar1aPATHTODLLMy.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further."_
Than it ran my tests? They are stored in that dll, it doesnt make sense... Ideally I wanted to do a build in the build pipeline, copy the bin folder, publish folder to artifact ->use that dll to run the rests in release.
With all the issues I changed it so
in build:

• I simply publish my whole repository into a artifact (container)

in release:

• I do a dotnet build on the .tests.csproj
• Run dotnet test (works, disabled it so I can try vstest)
• Run vstest with all settings you mentioned.

Edit: I switched it back to my original comment. Currently I don't have the same issue where it would say im using NetCoreApp v1, But I still am getting the _NQAP.ATC.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further._ And it does not run the tests.

[juarola]

@acesiddhu Regarding your answer to @joshbouganim "...Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file."

We are experiencing the same issue as @joshbouganim and i'm a bit confused about the "things remain the same" part. Is there something that should be done so the coverage results would be shown on the coverage tab? Currently there's just the option to download .coverage file.

[ghost]

@acesiddhu Regarding your answer to @joshbouganim "...Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file."

We are experiencing the same issue as @joshbouganim and i'm a bit confused about the "things remain the same" part. Is there something that should be done so the coverage results would be shown on the coverage tab? Currently there's just the option to download .coverage file.

Same here! I was expecting some awesome graphs =(

[joshbouganim]

@joshbouganim the old UI used to show Block level data as well. we have removed that because we feel user generally needs and interpret Line level data. Dealing with Block is what we think is hard (specially for large code base). Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.

so the new UI is the equivalent of the line metric from the old UI?

[karanjitsingh]

@joshbouganim the old UI used to show Block level data as well. we have removed that because we feel user generally needs and interpret Line level data. Dealing with Block is what we think is hard (specially for large code base). Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.

so the new UI is the equivalent of the line metric from the old UI?

@joshbouganim, yes that's correct

[RusmanCool]

Could VSTS just make it work with SonarQube? SonarQube is the best tool for not only Code Coverage but for defect idetifications, what VSTS did not even have.

[PBoraMSFT]

@carrocon - ack. Coverage reporting in the web (without needing to download files for offline viewing in the IDE) is on the backlog. There are two parts to the .NET core problem - coverage collection on all platforms and reporting. Our first priority will be to enable collecting coverage data on Linux and then do the reporting bit.

I am also locking the conversation here since the original issue is unrelated to the recent comments about coverage reporting - feel free to look through other related issues or open a new issue for this feedback.