Azure-pipelines-tasks: Write a warning when running the VSTest task and no tests were found

[roizentner]

Type: Feature
Enter Task Name: VSTest

Environment

• Server - Azure Pipelines
• Agent - Private (Windows, v2.141.1)

Issue Description

Hi,
There used to be a warning written when VSTest task did not find any tests to run.
This warning is no longer being written to the log.

Please bring it back...
This issue was born from the feedback received in https://developercommunity.visualstudio.com/content/problem/690316/vstest-task-behavior-change.html

[ShreyasRmsft]

We are working on revamping the experience of what happens when no tests are run. Will keep you updated once we spec out the plan.

[TysonMN]

@ShreyasRmsft, would be possible for that improved experience to include a fix for https://github.com/microsoft/vstest/issues/2217?

[ShreyasRmsft]

@bender2k14 this change is in the vstest task for AzureDevOps and will not have any bearing towards running tests via the dotnet cli.

[TysonMN]

Is that issue about running tests via the dotnet CLI? I thought it was (now) about running tests with VSTest.

I originally posted https://github.com/dotnet/cli/issues/12842, which _is_ about running tests via the dotnet CLI, but it was copied to https://github.com/microsoft/vstest/issues/2217.

[ShreyasRmsft]

@bender2k14 dotnet cli internally uses the vstest platform.

So does the VsTest task. This feature is being implemented in the AzureDevOps task and not the vstest platform.

[ShreyasRmsft]

This feature has been implemented in the vstest v2 task, please check the task docs.

[TysonMN]

Here is the link to the VSTest task. I think the relevant part of the documentation is

minimumExpectedTests Minimum # of tests | (Optional) Specify the minimum # of tests that should be run for the task to succeed. Total tests run is calculated as the sum of passed, failed and aborted tests. Default value: 1
-- | --

Thanks @ShreyasRmsft for your help with this :)

[TysonMN]

Finally confirmed this behavior. Here is another relevant setting.

failOnMinTestsNotRunFail the task if a minimum number of tests are not run | (Optional) Use this option to fail the task if a minimum number of tests are not run. This may be useful if any changes to task inputs or underlying test adapter dependencies lead to only a subset of the desired tests to be found.Default value: False
-- | --

Key thing to see there is that the default value is false. After setting it to true, I got an expected error.

[error]The specified minimum number of tests 1 were not executed in the test run.

Thanks again @ShreyasRmsft :)

[ShreyasRmsft]

Sadly the yml view does not show inter input dependency the way the designer view does. For the vstest task my guidance is to author via UI and export as yml or use the sidebar in the yml view.

[TysonMN]

Ah, yes. When I was new to the YAML, I used the designer view to generate the YAML I wanted. Thanks for the reminder about this approach.