Sonar-dotnet: Code Coverage: exclusions in .runsettings file are ignored

[berndku]

Description

after upgrading SonarQube Server from 5.6 to 6.7.4.38452 and C# plugin to 7.2 (build 5463), Code Coverage is still imported from VSTS build, but provided exclusions in .runsettings file for VSTest are ignored. With SonarQube 5.6 the .runsettings file exclusions were correctly applied.

Repro steps

VSTS SonarQube Build tasks used.
VSTest task has Code Coverage enabled and a .runsettings file specified in the property "Settings file".

Expected behavior

Code Coverage in SonarQube should recognize exlusions from VSTest .runsettings file.

Actual behavior

Code Coverage is imported into SonarQube, but exclusions are not recognized.

Known workarounds

Related information

• SonarC# Version : 7.2
• Visual Studio Version : 15.7.3
• Scanner for MSBuild: VSTS Tasks Version 3

[rliberoff]

We are having exactly the same issue.

Please fix it!!!!!!

[millusdk]

Are there any updates on this? We are experiencing the same issue in 6.7

[ghost]

Has anyone looked into where this change would need to be made? I would like to see if I can get this fixed/added, but I can't seem to locate where.

[Evangelink]

Hi there,

I finally managed to spend some time looking at this issue.

Starting with the version 6.6 of SonarC#, we are now processing a new metric called "executable lines of code" which is used in addition to the coverage info to track lines covered/uncovered. As we don't process (and we never did) the .runsettings file, all files excluded in this configuration won't have any coverage info BUT using the executable lines of code will appear as un-covered on SonarQube.

As a short-term answer, you will need to provide a new parameter (sonar.coverage.exclusion=LIST_OF_FILES) to the scanner for msbuild (or the prepare task of TFS/VSTS) OR configure your project on the SonarQube/SonarCloud project administration page to exclude all files you have excluded in your .runsettings file. Note that you can only exclude a file or a group of files BUT not something with a lower granularity.

As a mid-term/long-term solution, we will investigate whether we should automatically exclude all files excluded in the .runsettings file.

[puddlewitt]

Closed?

[andrei-epure-sonarsource]

Before moving forward with this, we'll have to check the impact of recently handling of .gitignore, and then come back with a plan

[JFMG]

@andrei-epure-sonarsource How far is the plan?

[StingyJack]

@andrei-epure-sonarsource - Would this apply to SonarCloud as well?

[andrei-epure-sonarsource]

I'm coming back on this topic after a long pause (sorry for that!).

@berndku @rliberoff @millusdk @puddlewitt @JFMG @StingyJack For clarification of your usecase(s): are you looking for excluding projects / files just for code coverage or from analysis in general? In other words:

• would your expectation be for us to read the runsettings.xml file and exclude those files completely from analysis (like the behavior @Evangelink mentioned here and in the docs, but out of the box based on what's excluded in the runsettings file)
• or would you expect these files to be analyzed by our analyzers, but excluded only from code coverage?

Thank you 🙏

---

Would this apply to SonarCloud as well?

@StingyJack - if this were implemented, yes, it would apply for both SonarQube and SonarCloud.

[berndku]

Analyzed, but excluded from code coverage.

[StingyJack]

Just excluded from code coverage analysis, which is what that runsettings file declares for the VS enterprise code coverage tooling.

[JFMG]

I agree with the previous comments. The runsettings file should just do what it says in the official documentation https://docs.microsoft.com/en-US/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file That means just exclude everything what is described in the runsettings file from code coverage.

[StingyJack]

At the bottom of that docs page @JFMG shared is a link to this page that details the types that can be included or excluded for coverage analysis.

In the past I've needed three of those filters.
ModulePath - to either whitelist the assemblies to check or to exclude unit tests from being analyzed for coverage
Attribute - to filter out the ones mentioned in the example runsettings
Source - to exclude code gen files by extension (*.g.cs)

[andrei-epure-sonarsource]

Thanks for the replies, @berndku , @JFMG , @StingyJack.

I'm thinking out loud: if we were to implement a partial solution (we like to deliver value incrementally) , for example starting with the Source filter only:

• would you find it more confusing that the Source filter is taken into consideration, and the ModulePath and Attribute (and others) are not?
• would you check the build logs in case the behavior from SonarQube/SonarCloud is different than what you'd expect (we'd mention in the build logs that only filter X is supported, and the rest are ignored)?

Another question: in my understanding, there would be only one .runsettings file per analysis, right?