Coverlet: No match found error when excluding multiple namespaces

[micheleissa]

Hi,
As suggested by @MarcoRossignoli I'm opening this issue after the short conversation on #56
When trying to use Exclude filter in MSBuild task integration to exclude more than one namespace I get some errors:

dotnet test --configuration Develop /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=TestResults/Coverage/ /p:Exclude=\"[*]CustomControls*,[*]TinyIoC*\"
zsh: no matches found: /p:Exclude="[*]CustomControls*,[*]TinyIoC*"

[MarcoRossignoli]

Seems related to escaping of double quotes...I don't know z shell but that error zsh: no matches found: looks like a shell error not a coverlet error.
Can you take a deep look on quotes escaping on that particular shell?We had similar issue with power shell https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/MSBuildIntegration.md#note-for-powershell--vsts-users
If you find a solution for zsh we can update guide

[micheleissa]

The same issue happens on AZDO pipeline so it has nothing to do with zsh
I don't get the part of using msbuild to run the tests? could you elaborate more please

Thanks.

[MarcoRossignoli]

The same issue happens on AZDO pipeline

Can you attach also AzDo error log?

I don't get the part of using msbuild to run the tests? could you elaborate more please

When you use /p:CollectCoverage=true you're underneath using msbuild(msbuild custom Tasks) integration, dotnet test run thanks to .net core version of msbuild.

Try to do a test with local sample

Clone this repo and go to https://github.com/coverlet-coverage/coverlet/tree/master/Documentation/Examples/MSBuild/MergeWith
Run this command to exclude all ClassLibrary2 namespace from tests

C:\git\coverletfork\Documentation\Examples\MSBuild\MergeWith (master -> origin)
λ dotnet test XUnitTestProject2\XUnitTestProject2.csproj /p:CollectCoverage=true  /p:CoverletOutput=../CoverageResults/ /p:Exclude="[*]ClassLibrary2*"
Test run for C:\git\coverletfork\Documentation\Examples\MSBuild\MergeWith\XUnitTestProject2\bin\Debug\netcoreapp3.1\XUnitTestProject2.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 0.8803 Seconds

Calculating coverage result...
  Generating report '..\CoverageResults\coverage.json'

+---------------+------+--------+--------+
| Module        | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary2 | 0%   | 100%   | 0%     |
+---------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 100%   | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 100%   | 0%     |
+---------+------+--------+--------+

We'll get 0% of coverage(that lib has got only one class for sample), in this case I ran on win terminal so without any escape.
Also I found on interenet that zsh: no matches found: is related to shell not to .net(sure not on coverlet) https://github.com/ohmyzsh/ohmyzsh/issues/31 like some chars interpretation of shell.

[micheleissa]

Ok, I will try to run the sample and report back here but it works fine for one excluded namespace the problem I'm facing is trying to exclude more than one namespace.
Here are the AZDO tasks I'm using per Microsoft documentation here Collect code coverage metrics with Coverlet section

    - task: DotNetCoreCLI@2
      displayName: 'dotnet test'
      inputs:
        command: 'test'
        projects: '**/*.UnitTests.csproj'
        arguments: '--configuration $(configuration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/  /p:Exclude=\"[*]TinyIoC*,[*]CustomControls*\"'
        publishTestResults: true


    - task: PublishCodeCoverageResults@1
      displayName: 'Publish code coverage report'
      inputs:
        codeCoverageTool: 'Cobertura'
        summaryFileLocation: '$(Build.SourcesDirectory)/**/coverage.cobertura.xml'

In the log attached line 1386 you see the following error

MSBUILD : error MSB1006: Property is not valid
Switch: [*]CustomControls*

I followed exactly the example mentioned in the docs that you specified to add escaped quotes around the string.
Test Task Output.log

I will be interesting to know how to get the command to work with zsh for a mac environment.

[petli]

zsh escaping is similar to sh or bash, you can find a maybe too detailed documentation here e.g.:
http://zsh.sourceforge.net/Guide/zshguide05.html

Double quotes usually disables globbing, but in your example there is \" which means that the quoting is escaped. It will then treat * as a globbing pattern, which fails since it of course doesn't match any files. You can try just removing the backslashes:

dotnet test --configuration Develop /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=TestResults/Coverage/ /p:Exclude="[*]CustomControls*,[*]TinyIoC*"

or if the double quotes are somehow needed for the msbuild task use single qoutes around the whole thing to preserve both double quotes and the pattern as-is:

dotnet test --configuration Develop /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=TestResults/Coverage/ '/p:Exclude="[*]CustomControls*,[*]TinyIoC*"'

[micheleissa]

That is what I get when I run the sample

[MarcoRossignoli]

ok that error is not related to coverlet it's related to sourcelinks(depends on how you've cloned/forked), copy/paste only MergeWith folder(top folder for instance c:\MergeWith in another folder cleanup generated bins files and re-run.
Thanks @petli

[micheleissa]

@MarcoRossignoli here you go, wonder why for me it says 100%

[MarcoRossignoli]

Enable log and check if filter is passed correctly https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md#msbuild-integration

[micheleissa]

Attached, it seems to be that it was passed correctly

[MarcoRossignoli]

Can you attach that Hits file... comment pls?

[micheleissa]

Can you attach that Hits file... comment pls?

I'm sorry I'm not following what is Hits file...

[MarcoRossignoli]

Sorry to you 😓 in left logging box last line of logs [coverlet] is something start with Hit file:/var... can you copy/paste all sentence?

[micheleissa]

[coverlet] Hits file:'/var/folders/9t/_l526qyn2z30vx9bh6ps2sp80000gq/T/ClassLibrary2_d0dfa25b-4216-44a0-b10b-06f907e976a9' not found for module: 'ClassLibrary2'

I navigated to that folder and definitely it is not there.

[micheleissa]

@MarcoRossignoli Also I wanna say thank you for the tremendous efforts you guys put in this tool :)

[MarcoRossignoli]

[coverlet] Hits file:'/var/folders/9t/_l526qyn2z30vx9bh6ps2sp80000gq/T/ClassLibrary2_d0dfa25b-4216-44a0-b10b-06f907e976a9' not found for module: 'ClassLibrary2'

Ok it's ok we skipped libs so no coverage file created...btw seems ok..I need to understand if that 100% is related to some round issue, can you attach coverage.json here?Should be "empty"

[micheleissa]

Had to change the extension so GitHub allows me to upload and yes it is empty.
coverage.txt

[MarcoRossignoli]

Ok it's working as expected, I'll take a look to console output to understand the difference.
Feel free to close if issue is solved!

[micheleissa]

But the issue is not resolved, I'm still not able to use comma separated exclusion list of namespaces. I attached above AZDO log.

[MarcoRossignoli]

Oh sorry again take a look at escape syntax like our build https://github.com/coverlet-coverage/coverlet/blob/master/eng/build.yml

Last test task
Does it work?

[micheleissa]

Oh sorry again take a look at escape syntax like our build https://github.com/coverlet-coverage/coverlet/blob/master/eng/build.yml

Last test task
Does it work?

Thanks, it finally worked with %2c suggestion and yes the test task works

[MarcoRossignoli]

Glad to hear!