Coverlet: Possibility to merge different test project results into operncover format

[balage90]

Currently if i understand correctly the /p:mergeWith paramter only works in case of Json format.
I'd like to have results in opencover output due to sonarqube alanizis.

We have multiple test for different csproj,-s in our project.

e.g.: dotnet test someProject.csproj --configuration debug /p:CollectCoverage=true /p:CoverletOutput=/TestResultOutput/Coverlet/SomeProject_Merge.xml /p:MergeWith=/TestResultOutput/Coverlet/Merge_All.xml /p:CoverletOutputFormat=opencover

Or if it already possible please correct me.

Br

Balázs

[petli]

If you don't need a merged opencover file, but just a merged coverage report of some kind, you can use https://github.com/danielpalme/ReportGenerator to read multiple opencover.xml files and produce a single report.

[tonerdo]

Hi @balage90, you can simply merge all your coverlet json result files and output them in OpenCover format

[balage90]

hi @tonerdo exactly thats the solution what i would like to achieve. But maybe i am using it in a wrong way. I could create the merged Json file but what is questionable for me how to "output them into open cover format". I am using dotnet test and for the each test execution it working fine to generate the json and insert it into the merged one. But how can i export it afterwards into open cover? If i understand correctly i had to convert the merged json at the end of the tests execution. Or you mean separetly generate open cover format for each?

[tonerdo]

Here's an example:

dotnet test /p:CollectCoverage=true /p:MergeWith="./other/result.json" /p:CoverletOutputFormat="opencover"

[pape77]

I'm trying to run this but since i'm doing a dotnet test on the solution which detects what to test, it will say that the result.json file does not exist, as it doesn't create it the first time it runs the test for one of the projects in the solution :/
I'm forced to run tests for one test proj and then run the test solution with the merge param, which results in two steps and double testing a project.

Also see my issue(#203) referring to my attempt to do this with the mergeWith arg pls

[balage90]

Yes i exactly wanted to do something similar, so for the first project test execution generates a merge.json and then i will try to use the MergeWith parameter at the second execution where I am referencing the merge.json from the previous run. And so on with the rest of the test projects. But sitll dont know how and wen should i able to generate opencover output since the previous repots are in Json format and not opencover.

[pape77]

@balage90 you should be able to do it in the final dotnet test excecution. So you have:

1- The first one just doing a dotnet test and generating the first coverage.json file
2- Subsequent n-2 dotnet test using the mergeWith parameter referencing the just created coverage.json file
3- Last one should also use mergeWith BUT ALSO /p:CoverletOutputFormat=opencover in order to generate the final opencover.xml file that you want, based on the merged coverage.json.

You can also just do a dotnet test on the first one to generate the coverage.json and then do a dotnet test on the solution with the opencover output param, to generate all the subsequent merges and the final opencover.xml generation (it's all in the issue #203)

The problem is, however, the one I mentioned in the issue I opened. I made a pr for fixing that, but has to be reviewed by @tonerdo yet

Let me know your findings, and if you face the same problem i had (since you use more than one test project too :) )

[balage90]

@pape77 Thanks a lot will try it out tomorrow and let you know the result.

[pape77]

@balage90 And? And? :P

[8]

I'm trying to run this but since i'm doing a dotnet test on the solution which detects what to test, it will say that the result.json file does not exist, as it doesn't create it the first time it runs the test for one of the projects in the solution :/

I am struggling with the same problem.
I've worked around that by creating an (almost) empty .json file to start with:

# create the testresults folder
$TestResultFolder = (Get-Location).Path + '\TestResults\'
New-Item -Path $TestResultFolder -ItemType Directory -Force

# create the initial file that coverlet needs for merging
$CoverletOutput = $TestResultFolder + 'coverage.json'
Set-Content -Path $CoverletOutput -Value "{}"

and then I can create a combined file like so:

$cmd = "dotnet test --no-build -c Release /p:CollectCoverage=true /p:CoverletOutputFormat=json /p:MergeWith=$CoverletOutput /p:CoverletOutput=$CoverletOutput src\Solution.sln --filter FullyQualifiedName~Unit --logger=trx -r $TestResultFolder"
Invoke-Expression $cmd

Now it's only 1 run over the whole solution and I got a combined coverage.json - but I am stuck now converting that file to opencover without supplying a test project.

Anybody got an idea? I haven't found any straightforward way to test a solution and produce a single coverage report.

Take care,
Martin

[pape77]

Hi @8

In #203 I create a opencover report.
Read my first comment for the commands.

You can replace the first run i make with your empty json file generation and it should work. Also i think the second step can be skipped and just run the 3rd one. Haven't tried it yet.

Problem comes with merging results (mentioned in the issue itself) . For what i created a pr #211 to fix this. You can read all about it in the issue and pr.

Cheers!

[8]

Hi @pape77,
thanks for the reply!

I finally solved my problem doing what @petli suggested:
1) I switched back to generating an opencover report per project in the solution by pointing coverlet at the solution file.
2) Then collect them using powershell and pass them as a single argument to reportgenerator which can handle multiple entries and combine them in a single report.

Take care,
-Martin

[pape77]

@8 ah I don't use the report generator or poweshell. I want coverlet to generate the merged coverage. Then it can be used for whatever.
That's what I'm aiming for with the pr ;)

But good you found a way for your case

Cheers

[vlef]

My pull request #220 is a quick fix that should allow for a way to accomplish this (see my comment), if someone's willing to test it.

[vlef]

Since #220 has been released, this issue should be closed. It is now possible to merge in other formats than json. Exlained in more details in issue #220