Reportgenerator: Totals on last entry in chart multiplied

[tr00st]

Having an issue using the Chart outputs showing history (ie: occurs on at least all of the HtmlChart, Html, and HtmlSummary standard report types). Duplicated with a stripped down output from OpenCover (occurs on the full version, stripped an output down to just our unit tests to give a relatively minimal reproducible case). Ran the conversion on the same coverage file (with ModuleTime incremented by one second being the only change), and it still occurs. Happy to send the coverage file, history and report outputs over privately assuming that would help.

On the chart outputs, the first point displays 1037/1285, 12/42 and 3055 and the last point displays 3111/3855, 36/126 and 9165 - the numbers are all multiplied by 3. Worth noting that on my main project the multiple here was 9.

Other numbers appear fine - the header sections of Html and HtmlSummary report correct totals (3055 lines) so this appears to be related specifically to history.

(Also have an issue regarding removing files from a project changes the historic count - ie: removing a 500 line file from the current project and running a report reduces the line count (and other stats) of old entries as if the file never existed - planning on dropping another report through on this once I've got an easily reproducible case together)

[tr00st]

Played with this a little further since the initial report - looks like it might be related somehow to the use of multiple report types?

Using:
%REPORTGEN_PATH%\ReportGenerator.exe -reports:coverage.xml -targetdir:coveragereport -historydir:coveragehistory -reporttypes:PNGChart;HTMLSummary;HTMLChart
Multiplies by 3

%REPORTGEN_PATH%\ReportGenerator.exe -reports:coverage.xml -targetdir:coveragereport -historydir:coveragehistory -reporttypes:PNGChart;HTMLSummary
Multiplies by 2

%REPORTGEN_PATH%\ReportGenerator.exe -reports:coverage.xml -targetdir:coveragereport -historydir:coveragehistory -reporttypes:HTMLSummary
Has the correct totals, but no history appears in the chart?

[danielpalme]

@tr00st
Thanks for reporting this issue. Somebody reported this some time ago (see #65)), but did not provide any details.
So thanks again for investigating.

I will have a look at this within the next days. Your second post contains some good observations.
Based on this information I should be able to reproduce the issue.

[danielpalme]

I found the problem with the multiplication problem and was able to fix it.

But I can't reproduce the last issue you mentioned:

%REPORTGEN_PATH%\ReportGenerator.exe -reports:coverage.xml -targetdir:coveragereport -historydir:coveragehistory -reporttypes:HTMLSummary
Has the correct totals, but no history appears in the chart?

Can you please share a sample summary report or a sample project which demonstrates the problem?

[danielpalme]

I just analyzed your email.

Your history files all have the same content (only date&time differ). Therefore only the first history point is rendered in the chart.
The second point in the chart contains the data from the current execution. It is rendered since the total lines from this run differ from the previous run(s) (it is 0 because your source code is no longer available).
So that seem pretty fine and as intended. And the chart is also visible in the HTMLSummary report.

The problem with numbers multiplied by x is already resolved.
I will take care of the remaining issue when classes are removed and historic coverage drops.

[danielpalme]

Maybe you want to try this new beta release:
https://www.nuget.org/packages/ReportGenerator/3.0.3-beta2

[tr00st]

Finally had a chance to take another look - 3.0.3-beta2 looks to have cleared that particular issue up - though I'm still seeing some inconsistencies on how the history is handled. It appears that applying an assembly filter to the report when generating historic reports applies only to the most recent state - ie: not when showing data from the coverage history files. Want me to open a separate case for that issue?

[danielpalme]

No need to create a separate issue for that.
I will have a look at it.

[danielpalme]

I took a look at the code.
You are right: When history of current execution is persisted to disk the filters on _assembly/class/file_ level are respected.
When reading the history files all assemblies/classes are read independently from the current filters.

So we are a talking about an edge case, that is only relevant if somebody changes the filters over time.
I believe that the current implementation is good enough, since:

• Filters are probably not changed very often
• When filters would be respected you can get other inconsistencies:
  
  
  
  • When you remove a "-" filter, the formerly excluded _assemblies/classes_ would not appear in history charts, since they did not get saved in the history file(s)
  
  
  • History reports only contain information on _assembly/class_ level. When you change filters on _file_ level those filters would not be respected as well.
  
  

A lot of changes would be required to cover all these edge cases. I don't think it's worth the effort.

[tr00st]

Not sure if it's user error, but that doesn't line up with my experiences. From what you've said, if an assemblyFilter is passed that excludes an assembly, it should not show up in the history generated by that run? From a quick test run here, the historic coverage files I'm generating contain historic coverage for assemblies that are excluded and not displayed in the detailed reports, ie:

%reportgenerator_exe% -reports:testOutput\coverage.xml -targetdir:testOutput\coveragereport -historydir:testOutput\coveragehistory -reporttypes:PngChart;Badges;CsvSummary;Html "-assemblyfilters:-Antlr3.Runtime;-Spring.*;-WOMSUnitTests"

Generates a full suite of reports (excluding Antlr3.Runtime, Spring and WOMSUnitTests), whilst the history generated from this command (after removing all other history) includes the excluded assemblies:

lang=xml <assembly name="WOMSUnitTests"> <class name="InTouch.RESTLibUnitTests.Serializers.CsvSerializerTest" coveredlines="72" coverablelines="72" totallines="0" coveredbranches="0" totalbranches="0" /> <class name="InTouch.RESTLibUnitTests.Serializers.JsonSerializerTest" coveredlines="43" coverablelines="43" totallines="0" coveredbranches="0" totalbranches="0" /> <class name="InTouch.RESTLibUnitTests.ValueProviders.RestQueryStringTest" coveredlines="82" coverablelines="86" totallines="0" coveredbranches="0" totalbranches="0" /> <class name="InTouch.WOMSUnitTests.BL.AccessRightTest" coveredlines="8" coverablelines="8" totallines="0" coveredbranches="0" totalbranches="0" /> <!-- more lines in here --> </assembly>

I'll try and see if I can nail down any particular cause on my end.

The caveats you've mentioned make sense otherwise - though some additional documentation on that or maybe a note on the command line help might be useful for future users.

[danielpalme]

You are right, the persisted history file contains all elements without filters applied.
I think I should change that behavior.

[danielpalme]

I published a new beta which persists the filtered assemblies/classes in the history files:
https://www.nuget.org/packages/ReportGenerator/3.1.0-beta3

[danielpalme]

Did you have a chance to try the new beta release?