Azure-pipelines-tasks: dotnet test eats quotes in TestRunParameters

[nohwnd]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: DotNetCoreCLI@2

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines or TFS on-premises?
  Azure pipelines

  • If using TFS on-premises, provide the version:

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

• Agent - Hosted or Private:

  • If using Hosted agent, provide agent queue name:

  • If using private agent, provide the OS of the machine running the agent and the agent version:

Issue Description

- task: DotNetCoreCLI@2
  inputs:
    command: 'test'
    arguments: '-- TestRunParameters.Parameter(name=%22EdiStagingFunctionsKey%22,value=%22key2%22) TestRunParameters.Parameter(name=%22EdiAADKey%22,value=%22key1%22)'

This is the only way I am able to pass TestRunParameters to the run. This was reported here: https://github.com/microsoft/vstest/issues/2387 and the escaping was fixed in vstest and dotnet sdk, but some additional rules are applied on the AzDO task side. Because for example \ is replaced by / and quotes are eaten by the task. So it is not true that what works on the command line works in the task (mentioned here https://github.com/microsoft/azure-pipelines-tasks/issues/10712#issuecomment-552195220 ) and none of the approaches described there worked for me, except for providing the quotes as URL escapes %22.

Ideally the user would be able to copy paste from the command line, where \" is the way to escape the quotes properly.

Task logs

https://dev.azure.com/jajares/params/_build?definitionId=6&_a=summary

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

[Insert error from the logs here for a quick overview]

[saquibkhan]

Hi, I have started looking at it.
So far my analysis is as below:
Issue Bug is with azure-task-lib _argStringToArray function - https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L88
Changing this function to provide the fix may require a lot of regression testing hence we are exploring other options to provide a mitigation fix in azure-pipeline-tasks

Stack-->
https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L88
https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L838
https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/DotNetCoreCLIV2/dotnetcore.ts#L170

Other Recommendations:
Fixing at the azure-pipeline-tasks level. Currently exploring this option.

[saquibkhan]

Below are our findings and next steps:

1. If we try passing quotes as a value even that is being eaten up by task-lib
   Input: -- TestRunParameters.Parameter(name=paramName,value=argwith"aquote)
   Output -- TestRunParameters.Parameter(name=paramName,value=argwithaquote)
2. If we use backslash then also quote is eaten up.
   Input: -- TestRunParameters.Parameter(name=paramName,value=argwith\"aquote)
   Output -- TestRunParameters.Parameter(name=paramName,value=argwith\aquote)

Other Inputs:
Input: -- TestRunParameters.Parameter(name=\"EdiStagingFunctionsKey\",value=\"key2\")
Output: TestRunParameters.Parameter(name=\EdiStagingFunctionsKey\",value=\"key2\")
Note: Above the first escaped quotes are eaten up.

Input: TestRunParameters.Parameter(name="EdiStagingFunctionsKey",value="key2"
Output: TestRunParameters.Parameter(name=EdiStagingFunctionsKey,value=key2)

Next Steps:

1. We will update the faq document to help our customers with the issue and a workaround.
2. azure-task-lib issue raised https://github.com/microsoft/azure-pipelines-task-lib/issues/648

[nohwnd]

https://github.com/microsoft/vstest/issues/862#issuecomment-621737720 I was exploring and fixing the vstest part here. It would be nice if the final syntax was compatible with these ways of escaping, as we don't know from whichOS the user will copy the command. ( Notice that on bash we escape parentheses and spaces.)

# cmd
dotnet test  -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\")

# powershell
dotnet test --%  -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\") 

# bash
dotnet test -- TestRunParameters.Parameter\(name=\"myParam\",\ value=\"value\"\) 

[saquibkhan]

yes @nohwnd once this issue https://github.com/microsoft/azure-pipelines-task-lib/issues/648 is fixed the escape sequence will work

[kcentralsquare]

I am experiencing this same issue with the --collect parameter when attempting to use coverlet as part of my build pipeline. URL escapes do not work.

[saquibkhan]

@kcentralsquare Can you try using %22 instead of quotes and see if that works. For time being this seems to be the only workaround here.

[kcentralsquare]

That did not work for me @saquibkhan.

[saquibkhan]

@kcentralsquare If it's possible can you share the command and argument?

[nehsharmMS]

@kcentralsquare are you still blocked on this issue or can we close this now?

[kcentralsquare]

@nehsharmMS Have you fixed the issue? I've switched to a script instead of using this task, but I would much prefer to use the task.

[jamierytlewski]

@saquibkhan I tried with the %22 and it's still not working. I was trying to use multiple formats for coverlet and was able to use %2c to change out the commas. I have tried this

'--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat="opencover,json,cobertura" /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ /p:Exclude="[*]Optimist.MrClean.*Service.*"'

And it "eats" up the quotes so the OutputFormat won't show up properly. I'm using a Linux instance to build

[ShreyasRmsft]

@jamierytlewski @kcentralsquare we are dependent on this issue https://github.com/microsoft/azure-pipelines-task-lib/issues/648 getting fixed. Could you please re-create that issue there since the old one got closed due to no activity.