Testfx: Cannot run tests anymore after upgrading to 2.2.1

[avivanoff]

Attempt to upgrade our test projects to version from 2.1.2 to 2.2.1 results in the following test execution error:

An exception occurred while invoking executor 'executor://mstestadapter/v2': Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

AB#1287803

[avivanoff]

A very simple sample project that reproduces the issue: UnitTestProject1.zip.

To get it to execute, just comment out DeploymentItem in UnitTest1.cs.

[kwilson-93]

I am also having issues successfully executing tests when upgrading from v2.1.2 to v2.2.1 (I am also using DeploymentItem mentioned above) 👍

[mprokopy]

Same for me.

[Sanan07]

You need to install this package Microsoft.TestPlatform.AdapterUtilities

@Haplois we need to think how to prevent this issue. Everyone who update MsTestAdapter to 2.2.1 will get this error.
Maybe somehow automatically download this package while updating MsTestAdapter

[mprokopy]

@Sanan07 for the following project, the installation of the package did not help:
UnitTestProject2.zip

[Haplois]

Hi @avivanoff and @mprokopy

Thank you for reporting this bug. There is an issue with the MSTest.TestAdapter package. Currently, you can disable app domains to make your tests functional.

<RunSettings>
  <RunConfiguration>
    <DisableAppDomain>True</DisableAppDomain>
  </RunConfiguration>
</RunSettings>

[avivanoff]

@Haplois, when do you expect to have a fix?

[Haplois]

We should have a preview in a week or two max, and if everything goes well, we can do another minor release.

[avivanoff]

@Haplois, is it a recommended approach to disable app domains in test execution?

[Haplois]

@avivanoff It should be fine, especially if tests are running in isolation.

[xantiva]

Hello,
after updating the MSTest.TestAdapter from v2.1.2 to v2.2.1, I can't run "all" tests either.

We have several test projects in the solution. I can run only some of them. For the others I got this errors in the test output:

Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxx.Device.Hardware.Adapters.Br.Tests.BrConnectionTests.Constructor|FullyQualifiedName=xxx.Device.Hardware.MeasurementUnits.Components.FilterWheel.Tests.ProtoBFilterWheelComponentTests.CleanupAsync|FullyQualif..." in "C:\source\xxx\Device.MU.PrototypeB.Tests\bin\Debug\netcoreapp3.1\xxx.Device.MU.PrototypeB.Tests.dll".

Sorry for the german error message, that should be something like: Incorrect format for TestCaseFilter Error: Missing ...

Ciao,
Mike

[Haplois]

@xantiva, can you clean your solution, close Visual Studio, reopen it, rebuild your entire solution and try running your tests again? Do you still get the error?

[Jehoel]

@Haplois I can also reproduce the issue and restarting VS after updating to 2.2.1 does not fix the issue, but downgrading back to 2.1.1 does fix it (no VS restart required).

In my case, I have two test projects: the first is targeting .NET Framework 4.8 with an old-style .csproj. The second project uses the same C# source files but targets .NET Core 3.1. If I update either project to 2.2.1 then that project will fail to run tests (but it discovers tests fine... weird).

When I update either test project project and try to run tests I get his output:

---------- Starting test discovery for requested test run ----------
========== Test discovery finished: 475 Tests found in 851.2 ms ==========
---------- Starting test run ----------
Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
No test matches the given testcase filter `FullyQualifiedName=MyProject.Tests.TurboencabulatorTests.ArrayExtensions_Chars_StartsWith_EndsWith_empty_strings_should_always_be_true|FullyQualifiedName=MyProject.Tests.TurboencabulatorTests.ArrayExtensions_Chars_StartsWith_EndsWith_nonempty_strings_s...` in C:\git\me\MyProject\MyTests-Net480\bin\Debug\MyProject.Tests.dll
========== Test run finished: 0 Tests run in 781 ms (0 Passed, 0 Failed, 0 Skipped) ==========

The output for my MyTests-NetCore31 project is the same (just with different paths, of course).

---

I also observed the same error as the OP a few times - it was repeated _for every test method_ in my .NET Core 3.1 Test project - but I can't seem to reproduce it anymore - which is weird. But it was something like this:

Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

Both of my test projects have a package-reference to Microsoft.NET.Test.Sdk, Version=16.9.1 (which in-turn has its own reference to Microsoft.TestPlatform.AdapterUtilities, Version=16.9.1) so I assume this is another case of a missing binding-redirect or an old reference that wasn't updated.

---

I also sometimes see this output:

---------- Starting test discovery ----------
Test project MyProject.Tests does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.
========== Test discovery skipped: All test containers are up to date ==========

...which is odd, because I thought it did. If I do a clean rebuild this message doesn't reappear.

---

Update:

After fiddling around with NuGet packages, I saw that if I updated MSTest.TestFramework from 2.1.1 to 2.2.1 (while still using MSTest.TestAdapter, Version = 2.1.1). The last line (starting with [MSTest][Discovery] is repeated for each test-method in the project (so about 400+ times, _lovely) in addition to the testcase filter error.

========== Test run finished: 0 Tests run in 1.4 sec (0 Passed, 0 Failed, 0 Skipped) ==========
---------- Starting test discovery ----------
Test project MyProject.Tests does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.
========== Test discovery skipped: All test containers are up to date ==========
---------- Starting test discovery for requested test run ----------
Test project MyProject.Tests does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.
Multiple versions of same extension found. Selecting the highest version.
  Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter : 14.0.5801.4
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.ArrayExtensionsTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery] (repeated for each test method)
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EnumParserTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EnumTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EnumTraitsCastingTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EnumTraitsTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EquatableEnumTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Enums.EnumsValues.EnumValuesTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.DateTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.DateTimeOffsetSerializationTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.DateTimeSetPartTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.DateTimeToStringIso8601Tests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.JsonTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.UtcDateTimeTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.DateTimeTests.DateTimeExtensionsTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Data.DataRecordExtensionsTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
[MSTest][Discovery][C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll] MSTestAdapter failed to discover tests in class 'MyProject.Tests.Data.ObjectDataReaderTests' of assembly 'C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll' because Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
No test matches the given testcase filter `FullyQualifiedName=MyProject.Tests.ArrayExtensionsTests.ArrayExtensions_Chars_StartsWith_EndsWith_empty_strings_should_always_be_true|FullyQualifiedName=MyProject.Tests.ArrayExtensionsTests.ArrayExtensions_Chars_StartsWith_EndsWith_nonempty_strings_s...` in C:\git\me\MyProject\MyProject.Tests\bin\Debug\MyProject.Tests.dll
Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
No test matches the given testcase filter `FullyQualifiedName=MyProject.Tests.ArrayExtensionsTests.ArrayExtensions_Chars_StartsWith_EndsWith_empty_strings_should_always_be_true|FullyQualifiedName=MyProject.Tests.ArrayExtensionsTests.ArrayExtensions_Chars_StartsWith_EndsWith_nonempty_strings_s...` in C:\git\me\MyProject\MyProject.Tests.DotNetCore\bin\Debug\netcoreapp3.1\MyProject.Tests.DotNetCore.dll
========== Test run finished: 0 Tests run in 1.6 sec (0 Passed, 0 Failed, 0 Skipped) ==========

[Haplois]

@Jehoel, thanks for the verification. You can use https://github.com/microsoft/testfx/issues/789#issuecomment-790097833 as a temporary workaround until we develop and test the fix.

[Jehoel]

@Haplois I've updated my earlier reply with more details and a more reliable repro of the other error.

Unfortunately the workaround you suggest won't work for me because I depend on AppDomains in the .NET Framework tests. I'll just stick with 2.1.1 for now:

    <!-- Don't update MSTest.TestAdapter or MSTest.TestFramework past version 2.1.1 until issues are fixed, see here: https://github.com/microsoft/testfx/issues/789 -->
    <PackageReference Include="MSTest.TestAdapter"   Version="(,2.1.1]" />
    <PackageReference Include="MSTest.TestFramework" Version="(,2.1.1]" />

[xantiva]

@Haplois here are my results:

The target framework of the solution is .NET Core 3.1.

Starting with MSTest.TestAdapter v2.1.2

• open Visual Studio v16.8.6
• load solution
• rebuild solution
• run all unit tests (of our main test category, the excluded tests use hardware) => ok
• Update MSTest.TestAdapter to v2.2.1
• rebuild solution
• run all unit tests => errors
• clean solution
• close Visual Studio
• reopen Visual Studio
• load solution
• rebuild solution
• run all unit tests => errors
• Revert MSTest.TestAdapter back to v2.1.2
• rebuild solution
• run all unit tests => ok

========== Test discovery finished: 683 Tests found in 13 sec ==========
---------- Starting test run ----------
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Device.Provider.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Provider.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Device.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\AppHost.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Device.Services.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Services.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Controlboard.Tests\bin\Debug\netcoreapp3.1\Controlboard.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Device.MU.PrototypeB.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.MU.PrototypeB.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Device.Shared.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Shared.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Hardware.Adapters.Hamamatsu.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Hardware.Adapters.Hamamatsu.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Shared.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Shared.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Utils.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Utils.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\WebApi.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.WebApi.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Hardware.Adapters.Schott.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Hardware.Adapters.Schott.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\WebApiMaintenance.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.WebApiMaintenance.Tests.dll".
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\Hardware.Adapters.Br.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.Device.Hardware.Adapters.Br.Tests.dll".
Ungültiges Format für TestCaseFilter Operator "|" oder "&" fehlt. Geben Sie das richtige Format an, und versuchen Sie es erneut. Beachten Sie, dass das falsche Format dazu führen kann, dass kein Test ausgeführt wird.
Kein Test entspricht dem angegebenen Testfallfilter "FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModuleTests.CheckAutoActivateForHardwareCommandScheduler|FullyQualifiedName=xxxxx.yyyy.Imager.Device.Bootstrapping.Tests.DeviceServicesServiceRegistrationModul..." in "C:\source\yyyy-Imager\yyyy-Imager-Win\source\yyyy-Imager\TestUtils.Tests\bin\Debug\netcoreapp3.1\yyyy-Imager.TestUtils.Tests.dll".
========== Test run finished: 0 Tests run in 12 sec (0 Passed, 0 Failed, 0 Skipped) ==========

[sumitkm]

Many thanks for the quick preview release. I can run unit tests with [DataTestMethod] attributes again using the preview.
I updated the MSTest.TestAdapter and MSTest.TestFramework packages from the preview feed and things seem to be back on track.

Will wait for 2.2.2 final release. Hopefully others who have reported this can verify it too.

Update: Oops just realised I cross posted, please refer to #793 for details of Preview release.

[Haplois]

@sumitkm, thanks for the verification.

[Haplois]

This bug is fixed in version 2.2.2-preview-20210310-03. The preview package is available in our daily feeds.

[xantiva]

@Haplois, thanks a lot. After updating to this preview version, I can run again my unit test!

[avivanoff]

@Haplois, when do you expect the official release with the fixes?

[Haplois]

@avivanoff testing on this is still continuing; we can do a servicing release on Monday if everything goes well.

[avivanoff]

@Haplois, any updates on the release?

[Haplois]

I preparing the release; I'll release it today.

[Haplois]

The version v2.2.2 release with the fix.

[avivanoff]

@Haplois, just updated to 2.2.2, the problem is not fixed. See updated sample project UnitTestProject1.zip.

Please, re-open the bug.

[xantiva]

Maybe this is an other problem? I can run my tests again after updating to v2.2.2. So my problem is solved with this release.

[mprokopy]

Can't run tests after update to 2.2.2. Please check the project from my post above. https://github.com/microsoft/testfx/issues/789#issuecomment-790006960

[Haplois]

@mprokopy, @avivanoff I missed the issue about DeploymentItem, diagnosing that too.

[Haplois]

Let's do this again :) a fixed version is just released. Please use v2.2.3.

[mprokopy]

Let's do this again :) a fixed version is just released. Please use v2.2.3.

Everything is fine on my side.

[StingyJack]

@Haplois - is part if this fix to report failures or at least some kind of inconclusive result for the tests we instructed the tool to run but it was not able to for whatever reason?

The fact that I commanded the program to do something and it failed to do that thing, and also failed to tell me there was a problem doing that thing seems like an obvious problem that should be fixed in any computer program, but I dont see that called out here.

[Haplois]

@StingyJack, the problem was caused by the parsing of test case filters, which has been addressed by the fix. Could you clarify if you're still having this issue?

[StingyJack]

I don't think the specific cause actually matters. It's a giant defect if any one part of a test framework can cause the rest to stop executing without reporting a single test failure

[Haplois]

That error prevented test discovery, So all the tests discovered on previous runs or via source-based discovery would show _not run_ status. You can configure Test Platform to report failure when no tests are discovered, but that's not the default behavior. See the documentation—example below.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <RunConfiguration>
    <!-- true or false -->
    <!-- Value that specifies the exit code when no tests are discovered -->
    <TreatNoTestsAsError>true</TreatNoTestsAsError>
  </RunConfiguration>
</RunSettings>

[StingyJack]

@Haplois There is a clear difference between "report an error when no tests are discovered" and "the tests that show in the test explorer list are not being executed when commanded to do so, and are not reporting any failure for the tests or any other failure to perform the command in its entirety."

If the test is in a Not Run state, then it has been discovered. The test is in the Test Explorer listing, I see it there, so it must be discovered at that point and known to the test runner. I tell VS to run that test and it tries to run it, but still it says "Not Run" (with the broken version of this package). Similarly the other tests in the list show as Success or Failure but they never change either. I expect them to at least be set to Not Run if I had selected them to run as part of the invocation and they did not run for whatever reason. In no way is it useful to anyone for them to continue to show the last completed run status.

What this leads to is that over the course of a few hours or days I would make changes to the target code and then Run all Tests and they would all show "Success" in test explorer and in the status bar. I believe that the refactoring changes I made were good, and then made more refactorings and other changes that needed to be made that depended on some refactoring. Except that the tests I commanded to be run werent actually being run. They were showing the status of the last time they were able to run. A gross misrepresentation of the actual state that was discovered late in the development effort that requires reworking and retesting the entire UserStory task that was in progress.

The summary of this is that your tool failed and did not report it to the operator. Like a train off the rails and the conductor is telling everyone "everything is fine and good and on schedule".

[Haplois]

@StingyJack, a discovered test doesn't necessarily mean that it was discovered by the test runner. It might have been discovered by the Test Explorer using source-based discovery or can be loaded from the cache. Your concerns are valid, but the Test Explorer window is not part of this repo.

Please report it inside the Visual Studio, using Send Feedback tool.

[StingyJack]

Your concerns are valid, but the Test Explorer window is not part of this repo.

What does the adapter do? What is it responsible for?