Navcontainerhelper: Create a code coverage report while running Run-TestsInNavContainer

[Sven-Niehus]

Hi,

it would be helpful to have the ability to create a code coverage report while running the tests with _Run-TestsInNavContainer_. The easiest way would be to add a option to the _Run-TestsInNavContainer_ commandlet (_-codeCoverage_ for example) that automatically generates the code coverage report alongside the XUnit test result file.

Having the code coverage for our tests would be really helpful for our build pipelines, because it would enable us to check for the code coverage in pull requests and thus automatically refuse bad pull requests that do not meet our quality expectations and still need to be worked on.

Azure DevOps supports code coverage files in Cobertura and JaCoCo format out of the box (Source [1] [2]), so those file formats would be preferred.

[Sven-Niehus]

An example for a Cobertura coverage file can be found here

[freddydk]

I think that is something the moderndev team is working on - and as soon as that exists, I will integrate it into the run-tests function.
@atoader - do you have any info about when we will have something like this?

[ShadowXVII]

Something we're keeping an eye out for too. Will help us ensure we comply with Microsoft AppSource code coverage requirements.

[atoader]

We are working on this, but I cannot provide an exact ETA. We should have it in place for Wave 2 2020 at the latest

[pmatsconsulting]

Hi,
Can we still expect have it in place for Wave 2 2020 at the latest?

[atoader]

Hi! The most likely timeline is CU2 of Wave 2 2020.

[ninten]

Hi! The most likely timeline is CU2 of Wave 2 2020.

Hi! Wave 2 2020 is already out, but what does CU2 mean?

[freddydk]

2020 wave 2 is 17.0
17.2 is what is referred to as cu2 here - december / january likely

[TheNCuber]

Now that BC 17.2 is out, are there any news concerning this functionality in navcontainerhelper?

[freddydk]

I will check up with the team to see if there is something we can use

[steinarb-ls]

Any updates on this?

[TheNCuber]

Any news on this @freddydk ? Code coverage in AL is available now through page 9990 I think, so I guess it should be possible to add it to BCContainerHelper. 🤔

[freddydk]

I haven't looked, but I guess it is just a matter of getting the time then...
Hopefully later in March

[elieassi]

Hello, any updates on this please?

[danielgorski-bynd365]

Hello again,
any updates regarding this feature?

[Spycoclown]

@freddydk how is the status?

[TheNCuber]

As I could not wait any longer, I basically wrote a PowerShell script (~600 lines) that parses the output of xmlports 9990 and 9991 (code coverage summary & code coverage detailed) as inspired by test-runner-service by @jimmymcp. It then generates an html report with an overview of the code coverage of all objects (screenshot 1) as well as a detailed html file for every object with line-by-line color coding and hit count, link to all procedures/triggers... (screenshot 2). The html reports are based on templates from the report generator that unfortunately doesn't support AL. The script also generates a very simple cobertura xml file with the global coverage percentage for Azure DevOps. I use the script in our Azure DevOps pipeline to publish the reports and the xml file for every build directly to Azure DevOps so the global percentage as well as the html reports are visible there.
At the moment the script is just a quick and dirty hack, but if I find time, I will try to open source a clean version of it, so we can maybe start from there.

Screenshot 1 (summary):

Screenshot 2 (detailed):

[Spycoclown]

@TheNCuber is it possible to get beta access? it looks great and I am looking for something like this since DevOps Pipeline can be used :)

[freddydk]

Very cool. It was my understanding that the code coverage didn't support the AL Test Runner (only the C/AL Test Runner, which will be removed any time). I will ask Nikola for his comments on this?