Pester: Code Coverage report counters are incorrect when there are multiple packages in the report.

[toddbrownmcis]

1. General summary of the issue

Code Coverage report counters are incorrect when there are multiple packages in the report.

2. Describe Your Environment

Pester version : 4.6.0 C:Program FilesWindowsPowerShellModulesPester4.6.0Pester.psd1
PowerShell version : 5.1.14409.1018
OS version : Microsoft Windows NT 6.1.7601 Service Pack 1

3. Expected Behavior

Expecting Code Coverage report counters to be the sum of all package counters.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE report PUBLIC "-//JACOCO//DTD Report 1.1//EN" "report.dtd">
<report name="Pester (02/11/2019 11:04:08)">
  <package name="Deploy/Common">
    <counter type="INSTRUCTION" missed="6" covered="67" />
    <counter type="LINE" missed="4" covered="51" />
    <counter type="METHOD" missed="2" covered="5" />
    <counter type="CLASS" missed="1" covered="3" />
  </package>
  <package name="Deploy">
    <counter type="INSTRUCTION" missed="0" covered="37" />
    <counter type="LINE" missed="0" covered="16" />
    <counter type="METHOD" missed="0" covered="1" />
    <counter type="CLASS" missed="0" covered="1" />
  </package>
  <counter type="INSTRUCTION" missed="6" covered="104" />
  <counter type="LINE" missed="4" covered="67" />
  <counter type="METHOD" missed="2" covered="6" />
  <counter type="CLASS" missed="1" covered="4" />
</report>

4.Current Behavior

Currently Code Coverage report counters are the same as the last package counters.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE report PUBLIC "-//JACOCO//DTD Report 1.1//EN" "report.dtd">
<report name="Pester (02/11/2019 11:04:08)">
  <package name="Deploy/Common">
    <counter type="INSTRUCTION" missed="6" covered="67" />
    <counter type="LINE" missed="4" covered="51" />
    <counter type="METHOD" missed="2" covered="5" />
    <counter type="CLASS" missed="1" covered="3" />
  </package>
  <package name="Deploy">
    <counter type="INSTRUCTION" missed="0" covered="37" />
    <counter type="LINE" missed="0" covered="16" />
    <counter type="METHOD" missed="0" covered="1" />
    <counter type="CLASS" missed="0" covered="1" />
  </package>
  <counter type="INSTRUCTION" missed="0" covered="37" />
  <counter type="LINE" missed="0" covered="16" />
  <counter type="METHOD" missed="0" covered="1" />
  <counter type="CLASS" missed="0" covered="1" />
</report>

5. Possible Solution

It seems like a report counter object needs to be created before package loop in Get-JaCoCoReportXml which accumulates counters from all package objects and is used when writing the report counters in Get-JaCoCoReportXml to the report.

6. Context

This is causing PowerShell project builds in TFS to publish incorrect Code Coverage results.

[DhruvM7]

My team is facing the same problem as well, in Azure DevOps. Glad to see there's a PR for this already. Is there any ETA for when this fix will become available?

[toddbrownmcis]

@DhruvM7 the PR is complete, just waiting for the repository owner to merge it into the master and build for release. I have no idea when that will happen.

As a workaround, I added a powershell task to my build after the Pester tests run and before the test results are published to update the counters in the Code Coverage report.

{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $true)]
        [System.String] $CodeCoverageOutputFile
    )

    # Read code coverage output file as xml document
    [System.Xml.XmlDocument] $xmlDocument = Get-Content -Path $CodeCoverageOutputFile

    # Update xml document
    [System.Xml.XmlNodeList] $packages = $xmlDocument.SelectNodes("/report/package")
    if ($packages.Count -gt 1)
    {
        [System.String] $type = ""
        [System.String[]] $types = @(
            "INSTRUCTION",
            "LINE",
            "METHOD",
            "CLASS"
        )
        foreach ($type in $types)
        {
            [System.Xml.XmlNode] $counter = $xmlDocument.SelectSingleNode("/report/counter[@type='" + $type + "']")
            [System.Int32] $counterCovered = 0 #[System.Int32]::Parse($counter.Attributes["covered"].Value)
            [System.Int32] $counterMissed = 0 #[System.Int32]::Parse($counter.Attributes["missed"].Value)

            [System.Xml.XmlNode] $packageCounter = $null
            [System.Xml.XmlNodeList] $packageCounters = $xmlDocument.SelectNodes("/report/package/counter[@type='" + $type + "']")
            foreach ($packageCounter in $packageCounters)
            {
                [System.Int32] $packageCounterCovered = [System.Int32]::Parse($packageCounter.Attributes["covered"].Value)
                [System.Int32] $packageCounterMissed = [System.Int32]::Parse($packageCounter.Attributes["missed"].Value)

                $counterCovered += $packageCounterCovered
                $counterMissed += $packageCounterMissed
            }

            $counter.Attributes["covered"].Value = $counterCovered
            $counter.Attributes["missed"].Value = $counterMissed
        }

        # Write updated xml document to code coverage output file
        [System.String] $xml = $xmlDocument.OuterXml
        $xml = $xml.Replace('"report.dtd"[]>', '"report.dtd">')
        $xml | Out-File -FilePath $CodeCoverageOutputFile -Encoding utf8
    }
}

[nohwnd]

I just merged it, I will release another beta today (or worst tomorrow) in the evening. :) Thanks for your contribution!