Azure-pipelines-tasks: dotnet test arguments removes quotes

[kyschouv]

Required Information

Type: Bug
Enter Task Name: DotNetCoreCLI
Task Version: 2.*

Environment

Server: Azure Pipelines
Agent: Hosted
Queue Name: Hosted Ubuntu 1604

Issue Description

The dotnet test task removes or changes quotes in the arguments list. Note how below, quotes and the escape character around the values for logger and /p:CoverletOutputFormat are removed.

Display name: Test
Command: test
Path to project(s): **/*Tests/*.csproj
Arguments: --configuration $(BuildConfiguration) /p:CollectCoverage=true /p:Exclude=[xunit.*]* /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ --logger:"console;verbosity=detailed" /p:CoverletOutputFormat=\"json,cobertura\"
Publish test results and code coverage: yes

Task logs

[command]/usr/bin/dotnet test /home/vsts/work/1/s/test/MyIntegrationTests/MyIntegrationTests.csproj --logger trx --results-directory /home/vsts/work/_temp --configuration Release /p:CollectCoverage=true /p:Exclude=[xunit.*]* /p:CoverletOutput=/home/vsts/work/1/s/TestResults/Coverage/ --logger:console;verbosity=detailed /p:CoverletOutputFormat=\json,cobertura"
MSBUILD : error MSB1006: Property is not valid.
Switch: cobertura

[sachinma]

@cltshivash can you look at this issue since this affects the test command of the dot net core cli task.

[CarlosLanderas]

I am having the same problem.

Original command:

/p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../../lcov

Task output:

/p:CollectCoverage=true /p:CoverletOutputFormat=\opencover,lcov" /p:CoverletOutput=../../lcov

[kyschouv]

I was eventually able to at least work around this, but without any documentation I was left reading the source code to see how line parsing worked, as well as searching the web for alternate solutions (as when I finally reached the correct format, e.g. \"json,cobertura\", it then didn't handle it correctly in the dotnet tool - it thought cobertura\" was a second argument).

Here's my workaround arguments:

--configuration $(BuildConfiguration) /p:CollectCoverage=true /p:Exclude=[xunit.*]* /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ "--logger:\"console;verbosity=detailed\"" /p:CoverletOutputFormat=json%2ccobertura /p:MergeWith=$(Build.SourcesDirectory)/TestResults/Coverage/coverage.json

This is a dotnet test task with multiple test projects btw.

[Mgamerz]

Having this same issue. Would be nice to see this fixed, I want to see code coverage but I want to be able to exclude libraries I have to build from source from my coverage test.

[ShreyasRmsft]

One guideline you can follow is that whatever works in the command line will work here. The same set of arguments in the exact same format.

Also note arguments to the 'test' verb for dotnet can be provided safely using quotes around every param this way:

dotnet test /p:"Param1" /p:"Param2"

You don't need to escape the outer quotes, only the inner ones. If you have inner quotes make sure you always put the outer quotes.

ex: dotnet test /p:"some arg \"with quotes\""

[ShreyasRmsft]

Try converting your arguments to this format, make sure they run on the local command line then try them out in the task. I believe documentation on this topic is scarce i'll see what i can do to enhance it.

But for now please let me know if these simple guide lines work for you.

[ShreyasRmsft]

Also if your command line is getting longer than 3-4 params i would recommend using a runsettings file to configure the test session. You can pass runsettings using -s|--settings .

Take a look at https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test?tabs=netcore21 and https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019

[ShreyasRmsft]

Closing this for now. Please re-open if the above guidance doesn't work for you.