Coverlet: A default value for the CoverletOutput MSBuild property

[alexanderkozlenko]

I assume that a default value for the CoverletOutput property should not be the $(MSBuildProjectDirectory) but the $(OutputPath). Here are a few reasons why this is a more appropriate value:

1. A code coverage report is a kind of program output, as well as binaries and XML documentation files.
2. A code coverage report will be automatically redirected to another location when the project stores output somewhere outside the project (or the solution).
3. Code coverage reports for different configurations can be different (e.g., due to #if directives). And thus, the value of $(OutputPath) can automatically store reports in different folders without overwriting since it evaluates to the $(BaseOutputPath)\$(Configuration)\ by default.

[eerhardt]

@alexanderkozlenko - why did you close this issue? I fully agree with your reasoning above - the default value shouldn't be in the source directory for a project. All the outputs should be directed to output folders.

[tonerdo]

@alexanderkozlenko I definitely see the sense in what you're proposing. My reasoning around the default location of the results is simply because it becomes much easier to find especially since Visual Studio (and possibly other IDEs) doesn't show the contents of the output folders in the Solution Explorer

[eerhardt]

@tonerdo - you could follow that same reasoning for any output file. The build .dll and .pdb. The documentation .xml file. The test results file .trx.etc. They would be easier to find in VS if they weren't in the "output" directory. However, most users know to look in the output directory for the outputs. So putting output files in a different directory actually makes it confusing.

Writing to the "source" directory of a project during the build is generally frowned upon. One reason is that people need to add a specific .gitignore entry for those files (I assume code coverage results files shouldn't be checked in). Another reason is that a human can easily tell what can be cleaned up - just delete the bin and obj folders.

I would imagine most projects would need/want to change this default, which doesn't make it a very good default in my opinion.

[alexanderkozlenko]

I assume another possible option for the output path might be the usage of VSTestResultsDirectory property.

[tonerdo]

@alexanderkozlenko what path does VSTestResultsDirectory resolve to?

[alexanderkozlenko]

@tonerdo, AFAIK the default path depends on the exact test framework used and is provided by --results-directory option of dotnet test command. For MSTest v2 it is $(MSBuildProjectDirectory)\TestResults. Usage of this property can consolidate the output path for all unit test outcomes (e.g., unit test log, code coverage report).

[MarcoRossignoli]

@vagisha-nidhi can you tell us where/how vs test "calculate" output path for coverage on collectors?We could align with it.

[MarcoRossignoli]

AFAIK the default path depends on the exact test framework used and is provided by --results-directory option of dotnet test command.

Now we support dotnet test with native collectors so we support --results-directory by design https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md#coverlet-integration-with-vstest

For MSTest v2 it is $(MSBuildProjectDirectory)\TestResults. Usage of this property can consolidate the output path for all unit test outcomes (e.g., unit test log, code coverage report).

Yep also vstest write results to MSBuildProjectDirectory so to be aligned a user can pass /p:CoverletOutput=TestResults\

For the moment doesn't seem a great issue, I mean we have a way to solve and also I don't know if follow test frameworks "defaults"(not only ms for instance) is a good solution for future.
I'll close this isssue for the moment, feel free to re-open if we want go on with discussion.