Vstest: Code Coverage can't be collected with codecoverage.exe

[smadala]

_From @maulwuff on February 16, 2018 8:31_

Description

Code coverage fails to be collected when used with codecoverage.exe
It succeeds when vstest.console.exe is used

Steps to reproduce

Try to run unit tests and collect code coverage from cmd (to be passed to SonarQube):

set CodeCoverage="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe"
set vsTestConsole="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
set coveragePath="%cd%\tests.coveragexml"

rem this one creates a emtpy coverage result:
%CodeCoverage% collect /output:"tests.coverage" %vsTestConsole% /EnableCodeCoverage "UnitTests.dll"

rem this one does create the coverage file as expected:
%vsTestConsole% /EnableCodeCoverage "UnitTests.dll"

In the event log you can find this error entry for the first try when using CodeCoverage.exe:

.NET Runtime version 4.0.30319.0 - Loading profiler failed during CoCreateInstance.  Profiler CLSID: '{9317ae81-bcd8-47b7-aaa1-a28062e41c71}'.  HRESULT: 0x80040111.  Process ID (decimal): 520.  Message ID: [0x2504]

And this info eventlog entry for the run with vsTestConsole on its own:

.NET Runtime version 4.0.30319.0 - The profiler was loaded successfully.  Profiler CLSID: '{9317ae81-bcd8-47b7-aaa1-a28062e41c71}'.  Process ID (decimal): 1092.  Message ID: [0x2507]

Expected behavior

• There should be no error from loading the profiler in the event log
• code coverage file should be populated as it is done with using vstest.console.exe on its own

Actual behavior

• see above.
• vstest.console.exe on its own runs fine
• vstest.console.exe used with codecoverage.exe fails to produce coverage info

Environment

win7 prof

Microsoft Visual Studio Enterprise 2017
Version 15.5.6
VisualStudio.15.Release/15.5.6+27130.2027
Microsoft .NET Framework
Version 4.7.02558

_Copied from original issue: Microsoft/testfx#371_

[smadala]

@maulwuff I believe this is not supported scenario.

/c @pvlakshm @cltshivash @mayankbansal018 is this supported scenario?

[maulwuff]

The steps are actually taken from https://docs.sonarqube.org/pages/viewpage.action?pageId=6389770

[mayankbansal018]

@maulwuff , CodeCoverage.exe can be used to gather coverage data, however it is not our primary supported scenario. We recommend folks to always use vstest.console to gather coverage.

Is there any limitation as to why you can't use vstest.console in your scenario?

[pvlakshm]

@maulwuff,
Agree with @mayankbansal018. Please use vstest.console to gather codecoverage.

[smadala]

@maulwuff Ping us if you find any limitation with collecting code coverage with vstest.console.exe

[gremlm]

When using code coverage with vstest.console.exe it is impossible to specify location of the output .coverage file! That problem makes usage of the code coverage with vstest.console.exe useless!