Vstest: Version 16.0.0 The given test sources target multiple frameworks/platforms that are incompatible

[bogdancucosel]

Description

WE have both .net core and standard tests.
in version 15.9.0 we could run them in a single VSTest task,
Now, in 16.0.0 it fails with:

The given test sources target multiple frameworks/platforms that are incompatible. Please make sure the sources target the same framework and platform.

In VS , running all the tests also works.
We would expect this to work in VSTS also.

[shortstuffsushi]

We don't have multiple targets; all of our projects are currently targeting .NET Framework 4.7.1, and we're also seeing this issue. We do have the .NET Core tools installed on the server, and other projects in other solutions do use them for what it's worth.

[cbrianball]

I started getting this error today as well. My solution has a single project:

<TargetFramework>net462</TargetFramework>
<RuntimeIdentifier>win7-x86</RuntimeIdentifier>

My unit tests are UI tests (Selenium), so the code it is testing is running on a different server.

If I force version 16.0.0-preview-20190203-03, then my tests run as expected. Only when targeting 16.0.0 do I get this error.

Looking back at previous logs, I do see output stating which assemblies are not compatible with my chosen framework, but this was information text (not even a warning):

Test run will use DLL(s) built for framework .NETFramework,Version=v4.6.2 and platform X86. Following DLL(s) do not match framework/platform settings.
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.TestFramework.dll is built for Framework 4.5 and Platform AnyCPU.

I tried to update my project to net45, but I have dependencies that only work with .NETStandard 2, so I can't do that. I also can't upgrade to .netcoreapp2.0 because the version of the TestContext that I get with .NET Standard doesn't allow me to add attachments.

For now I can work around by forcing the most recent preview version of the tools.

[Suprndm]

Since Yesterday we have the same issue :
Visual Studio Test Platform Installer 1.* on latest
Visual Studio Test : Installed by Tools Installer

The given test sources target multiple frameworks/platforms that are incompatible.

[error]VsTest task failed.

All projects are in .net 4.7.

For now forcing the Installer on Specific Version 15.9.0 solved the issue.

[vagisha-nidhi]

We have a breaking change in 16.0.0 regarding running tests targeting multiple framework dlls and incompatible target framework given by the user which is mentioned in the release notes here.
According to the associated changes, (1) multiple dlls targeting different frameworks (version irrespective) will throw a framework/architecture incompatible error.
Explanation : if there is one dll targeting .NETFramework and the other targeting .NetCore or vice-versa, the run would be aborted. Earlier we used to spit out warnings and run tests from one of the target dlls (skipping the other dlls).
The scenario where all dlls are of either .NetFramework or .NETCore (even if versions are different) should work seamlessly as before.
(2) The other change we made was that if a dll is .NetCore and user explicitly sets /Framework to .NETFramework or dll is .NETFramework and /Framework is set to .NetCore(i.e. frameworkcore10), this gets aborted due to mismatch between given dlls and user provided input.

@bogdancucosel Can you please check if your scenario is the one I mentioned first? That is you are targeting both .NetFramework and .NetCore test dlls which would break?

@shortstuffsushi This shouldn't break if all projects are targeting .NET Framework 4.7.1. Can you please check if your scenario is the one I mentioned first? If not, can you please provide a sample repro project?

[bogdancucosel]

@vagisha-nidhi yes, you are right. It seems we have misread the test result before. We now have 2 VSTest tasks, targeting each framework individually , and everything works as expected. (we were also confused because the release notes were not posted together with the 16.0.0 release ). As far as we are concerned, the issue can be closed.

Thank you very much,
Bogdan

[cbrianball]

@vagisha-nidhi I don't believe the explanation you provided addresses my issue. I provided details in my first post on this issue.

Short version: My tests target .NET 4.6.2. I do have some dependencies that target .NET Standard (NOT .NET Core).

[mmyk]

I am also seeing this issue, and my tests target .NET 4.7.2, and all dependencies target versions of .NET Framework except one which targets both .NET 4.7 and .NET Standard.

[vagisha-nidhi]

The change we made for aborting test run when there are multiple targets was failing on CI because the vstest.console task was picking up Adapter dlls(present in output folder), that is, TestPlatform.MSTestAdapter.PlatformServices.Interface.dll and TestPlatform.MSTestAdapter.PlatformServices.dll since default task template provides test*dll as pattern, of which one target .NetPortable and the other .NetStandard.
We have reverted the change and released Test Platform 16.0.1 with the revert.
Can you please check if you are still facing the issue with 16.0.1?

[Emil-H]

My configuration was similiar to @mmyk (only .NET 4.7). Using version 16.0 failed with the reported error message.
Using version 16.0.1 solved it.

[cbrianball]

16.0.1 resolve my issue as well.

[mmyk]

@vagisha-nidhi Using 16.0.1 resolved the issue for me.

[vagisha-nidhi]

Closing the issue as 16.0.1 worked.

[nmfernandes]

I am still facing this issue with version 16.0.1.

`Microsoft (R) Test Execution Command Line Tool Version 16.0.1
Copyright (c) Microsoft Corporation. All rights reserved.

vstest.console.exe
"D:\a\r1\a_PEG UI Tests Build\drop\E2E_test_automation\bin\Release\E2E_test_automation.dll"
"D:\a\r1\a_PEG UI Tests Build\drop\E2E_test_automation\bin\Release\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll"
"D:\a\r1\a_PEG UI Tests Build\drop\E2E_test_automation\bin\Release\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll"
"D:\a\r1\a_PEG UI Tests Build\drop\E2E_test_automation\bin\Release\Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll"
"D:\a\r1\a_PEG UI Tests Build\drop\E2E_test_automation\bin\Release\Microsoft.VisualStudio.TestPlatform.TestFramework.dll"
/logger:"trx"
/TestAdapterPath:"D:\a\r1\a"
/Framework:.NETFramework,Version=v4.6.2

Starting test execution, please wait...
Test run will use DLL(s) built for framework .NETFramework,Version=v4.6.2 and platform X86. Following DLL(s) do not match framework/platform settings.
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll is built for Framework 4.5 and Platform AnyCPU.
Microsoft.VisualStudio.TestPlatform.TestFramework.dll is built for Framework 4.5 and Platform AnyCPU.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.`

[anaprikho]

@nmfernandes
have you solved this issue?

[nmfernandes]

@anaprikho Although those messages about "DLL(s) do not match framework/platform settings" keep being displayed, the tests run as expected, so after all this is not an issue for me.

[anaprikho]

@nmfernandes can you see then the results of your tests? It seems there is no test available cause of the platform incompatibility

[nmfernandes]

@anaprikho, and are you sure that the platform incompatibility is the reason for which the test results are not being displayed?