Cake: OpenCover tool only intercepts the last test assembly

[jenshorn]

Internally OpenCover intercepts the test alias and runs it. The problem however is that the test alias only executes once, for the last test-dll even if there are multiple assemblies found by the globber.

So when running a test runner separately all matching test assemblies are processed...

Task("Run-Unit-Tests")
    .IsDependentOn("Build")
    .Does(() =>
{
    XUnit("./src/**/bin/" + configuration + "/*.Tests.dll");
});

But when running the same test runner within OpenCover, the same test assemblies are found but only the last one is intercepted and processed.

Task("Run-Code-Coverage")
    .IsDependentOn("Build")
    .Does(tool =>
    {
        tool.XUnit("./src/**/bin/" + configuration + "/*.Tests.dll");
    },
    new FilePath("./result.xml"),
    new OpenCoverSettings()
);

The same behavior is observed both with MSTest and XUnit. Perhaps because both of them iterates over the collection and executes the test assemblies one by one?

[patriksvensson]

This should remain open since this still applies to xUnit.

[gep13]

Ah, gotcha. Should we make that a separate issue? Or will we be able to correct that before we ship 0.9.0?

[patriksvensson]

@gep13 Hmmm. You've got a point. What do you think is best?

[gep13]

@patriksvensson I would be happy to leave it open, and complete the XUnit work under this same issue, but at the same time, I wouldn't want to hold up the release because it wasn't in. Is there a lot of work to correct the XUnit side of things?

[patriksvensson]

@gep13 No, not much to do. I can fix it tonight to get it out of the way :smile:

[gep13]

@patriksvensson that sounds perfect :+1:

[bjorkstromm]

DotCover #691 also suffers from this issue since it works the same way as OpenCover.

[patriksvensson]

@mholo65 This should be fixed in v0.10 for dotCover as well :smile:

[jenshorn]

Should this be reopened due to #745?

[patriksvensson]

@phrusher xUnit 1 do not seem to support multiple assemblies so not sure what can be done here.

[devlead]

@phrusher xunit 2 still supports multiple assemblies, was only the old XUnit V1 aliases that were reverted

[devlead]

@patriksvensson exactly, as XUnit2 is backwards compatible it should suffice.

[jenshorn]

@patriksvensson @devlead Just wanted to check as it related to this issue. We are running XUnit1 but I guess it might be difficult to fix this issue completely.

[patriksvensson]

@phrusher You can use the xUnit v2 tool to run xUnit v1 assemblies :smile:

[devlead]

@patriksvensson yeah that would work, could be an issue if tests depend on xunit pre 1.9.2 it could potentially be some breaking changes, but worth a shot ;)
@phrusher One way would be to make the XUnit 1 alias smarter, but that would likely be served best as a separate issue non related to OpenCover.

[jenshorn]

@patriksvensson Well I'll be damn.. It worked! :)
@devlead As the XUnit2 tool seem to run fine we can leave XUnit1 as is. At least until someone else stumbles over an issue with it :)

Thanks!

[AnimationChen]

I copy your code, running is fail!

[patriksvensson]

@AnimationChen What code? The code at the top represents the original issue posted 3,5 years ago and will not work.