Azure-pipelines-tasks: Merge coverage reports from multiple runs

[pkuczynski]

Required Information

Type: bug
Enter Task Name: PublishCodeCoverageResults@1

Environment

• Server - Azure Pipelines

Issue Description

Task PublishCodeCoverageResults@1 supports reports from multiple files, but the summary just shows the first match.

Task logs

The behavior for a task configured with a wildcard path for the coverage file is a little inconsistent:

      - task: PublishCodeCoverageResults@1
        inputs:
          codeCoverageTool: cobertura
          summaryFileLocation: $(System.DefaultWorkingDirectory)/packages/*/build/coverage/coverage.cobertura.xml

In this example, there are few coverage.cobertura.xml files from separate test runs. Unfortunately, I receive warning:

```

[warning]Multiple file or directory matches were found. Using the first match: ```

Ideally, the summary would show the coverage for each assembly separately. Because I'm working in a monorepo and would like to trace multiple folders separately.

A similar issue has been reported in https://github.com/MicrosoftDocs/vsts-docs/issues/4135

[mhamilton723]

+1 on this issue!

[pderaaij]

This would be a great addition when working with a monorepo.

[MisinformedDNA]

Is there a workaround for this?

[pkuczynski]

Not that I know of :(

[juliocr-ciandt]

+1

[boyanio]

I use a custom script to merge all reports into a single one. In this gist you can see an example using istanbul

[avilde]

I use jest-junit and cobertura reports both from front-end (create react app, jest) and back-end (.NET Core 3.0) but there is no merge option for both reports.
What a pity.

+1

[jhonnymertz]

Is there any update on this issue? I'm working on a monorepo and the fact that it cannot collect all the coverage reports limits the usage of this information in dashboards, for example. All the existing widgets to report code coverage in ADO dashboards rely on this data...

[MisinformedDNA]

To get around this, we can merge all the files using ReportGenerator. This is what worked for me:

1. (Optional) If the code coverage is coming from different jobs, publish the coverage files as artifacts and then have a later job download all the artifacts

2. Create a report from all coverage files and publish

- task: reportgenerator@4
  inputs:
    reports: **/coverage.cobertura.xml;artifact/**/cobertura-coverage.xml
    targetdir: $(Pipeline.Workspace)/coveragereport

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: "Cobertura"
    summaryFileLocation: $(Pipeline.Workspace)/coveragereport/Cobertura.xml

This has worked well for me.

[andreujuanc]

Work around is good and all, but in some orgs, it can take quite a while until you get a marketplace extension approved, if at all.
100% should be handled by Azure Pipelines and not some random extension.

[drdamour]

fwiw it looks like this might have been fixed with the v2 of this task at https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/PublishCodeCoverageResultsV2/task.json with https://github.com/microsoft/azure-pipelines-tasks/pull/11144 but it hasn't been published to devops...not sure why was also asked in #13257

[drdamour]

@MisinformedDNA where's that reportgenerator task come from?

[MisinformedDNA]

@drdamour https://marketplace.visualstudio.com/items?itemName=Palmmedia.reportgenerator