Coverlet: Coverage report is not generated if there are any failing tests in the run scopes

[pms1969]

More of a question really, but I've noticed that a test run with a failed test in it does not produce any coverage summary. Is that expected? If so is there some way to change that behaviour?

I'm going to go and check if it does produce the coverage file. If it does produce that, it's not really an issue for me, but would be nice to have the behaviour synced.

Thanks.
Paul

[tonerdo]

That's expected, because it's MSBuild integrated when the test fails MSBuild terminates the process and targets supposed to run after tests are not triggered

[pms1969]

OK. Thanks.

[pms1969]

Actually, is there any way to change that? We're running coverlet in our build pipeline with sonarqube. seeing the coverage drop suddenly is going to distract everyone from the real problem which would be the test failure. Code is still covered, even if one of the tests fail, even if it is a different from the happy path.

[tonerdo]

At this point, no. When I create an alternative console tool mode for Coverlet (much like how Opencover works by default) which isn't tied to the build system, then yes.

[pms1969]

ok. Thanks.

[chris5287]

@tonerdo could you document this behavior somewhere, as it took us a while to work out why no coverage file was being generated (thanks for fab project!)

[janaka]

+1 to running coverage even when tests fail.

[janaka]

@pms1969 mind changing the title of this issue to something like "Coverage report is not generated if there are any failing tests in the run scope"

Note that if you filter test execution to one(s) that pass coverage is generated.

[pms1969]

No problem. Will do when I get to work.

[tonerdo]

@pms1969 @janaka showing coverage results even when builds fail won't be supported because of how MSBuild integrations work. MSBuild will not call the next task if the previous task returns a non-zero code. #143 introduces a global tool that doesn't have this limitation, will be released Friday

[tonerdo]

@pms1969 @janaka @chris5287 You won't have this problem when you use the .NET Core global tool https://www.nuget.org/packages/coverlet.console/

[pms1969]

@tonerdo champion. thank you.