Vstest: dotnet test for multi target net461 and netcoreapp3.1 success in windows but fail in Ubuntu 20.04 with error: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel

[moh-hassan]

Environment

VS code
Version: 1.46.1
OS: Windows_NT x64 10.0.19041
Working in WSL 2 environment (Ubuntu 20.04, Mono is installed)

Steps to reproduce

A multi-target sdk-style Xunit test project

<TargetFrameworks>net461;netcoreapp3.1</TargetFrameworks>

Package reference:

<ItemGroup>
    <PackageReference Include="FluentAssertions" Version="5.4.1" />    
    <PackageReference Include="xunit" Version="2.4.0" />    
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" /> 
   <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.6.1" />    
  </ItemGroup>

I followed the instructions Getting Started with xUnit.net Multi-targeting with non-Windows OSes

The test success locally in windows 10 x64 using vs code:

dotnet test

When running the test remotly in WSL 2 environment it fail for net461 with error:

Could not load type of field 'Xunit.Runner.VisualStudio.VsExecutionSink:recorder' (4) due to: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

If I add the next reference, test success

<PackageReference Include="Microsoft.TestPlatform.ObjectModel" Version="16.6.1" />  

Should I add reference to Microsoft.TestPlatform.ObjectModel when runing in Linux environment in net461?
What the role of this package?

[nohwnd]

I tried to reproduce and did not see the issue. Is there anything more to how you build / run the tests?

[moh-hassan]

Can you try to run test for this project
Only I use 'dotnet test' from terminal in vscode when remotely in wsl 2.

[moh-hassan]

Also, I run dotnet test in native Ubuntu 18.04.2 LTS and the same error in net461 is fired.
here more details (the same in wsl2 and native Ubuntu client)

Click to expand Server stack trace!

Server stack trace:
at (wrapper managed-to-native) System.RuntimeTypeHandle.type_is_assignable_from(System.Type,System.Type)
at System.RuntimeTypeHandle.CanCastTo (System.RuntimeType type, System.RuntimeType target) [0x00000] in :0
at System.RuntimeType.IsAssignableFrom (System.Type c) [0x00020] in :0
at System.RuntimeTypeHandle.IsContextful (System.RuntimeType type) [0x00000] in :0
at System.RuntimeType.IsContextfulImpl () [0x00000] in :0
at System.Type.get_IsContextful () [0x00000] in :0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef, System.Boolean fRefine) [0x00041] in :0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef) [0x00000] in :0
at System.Runtime.Remoting.Messaging.CADMessageBase.UnmarshalArgument (System.Object arg, System.Collections.ArrayList args) [0x0003d] in :0
at System.Runtime.Remoting.Messaging.CADMessageBase.UnmarshalArguments (System.Object[] arguments, System.Collections.ArrayList args) [0x00011] in :0
at System.Runtime.Remoting.Messaging.CADMethodCallMessage.GetArgs (System.Collections.ArrayList args) [0x00000] in :0
at System.Runtime.Remoting.Messaging.MethodCall..ctor (System.Runtime.Remoting.Messaging.CADMethodCallMessage msg) [0x0001e] in :0
at System.AppDomain.ProcessMessageInDomain (System.Byte[] arrRequest, System.Runtime.Remoting.Messaging.CADMethodCallMessage cadMsg, System.Byte[]& arrResponse, System.Runtime.Remoting.Messaging.CADMethodReturnMessage& cadMrm) [0x00012] in :0
at (wrapper remoting-invoke-with-check) System.AppDomain.ProcessMessageInDomain(byte[],System.Runtime.Remoting.Messaging.CADMethodCallMessage,byte[]&,System.Runtime.Remoting.Messaging.CADMethodReturnMessage&)
at System.Runtime.Remoting.Channels.CrossAppDomainSink.ProcessMessageInDomain (System.Byte[] arrRequest, System.Runtime.Remoting.Messaging.CADMethodCallMessage cadMsg) [0x0000d] in :0

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke (System.Runtime.Remoting.Proxies.RealProxy rp, System.Runtime.Remoting.Messaging.IMessage msg, System.Exception& exc, System.Object[]& out_args) [0x0014d] in :0
at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_remoting_wrapper(intptr,intptr)
at (wrapper remoting-invoke) Xunit.Sdk.TestFrameworkExecutor1[Xunit.Sdk.IXunitTestCase].RunTests(System.Collections.Generic.IEnumerable1,Xunit.Abstractions.IMessageSink,Xunit.Abstractions.ITestFrameworkExecutionOptions)
at (wrapper xdomain-invoke) Xunit.Sdk.TestFrameworkExecutor1[Xunit.Sdk.IXunitTestCase].RunTests(System.Collections.Generic.IEnumerable1,Xunit.Abstractions.IMessageSink,Xunit.Abstractions.ITestFrameworkExecutionOptions)
at Xunit.Xunit2.RunTests (System.Collections.Generic.IEnumerable1[T] testCases, Xunit.Abstractions.IMessageSink messageSink, Xunit.Abstractions.ITestFrameworkExecutionOptions executionOptions) [0x0000e] in <4d712162c0a34cb99c967a7a106084df>:0 at Xunit.XunitFrontController.RunTests (System.Collections.Generic.IEnumerable1[T] testMethods, Xunit.Abstractions.IMessageSink messageSink, Xunit.Abstractions.ITestFrameworkExecutionOptions executionOptions) [0x00006] in <4d712162c0a34cb99c967a7a106084df>:0
at TestFrameworkExtensions.RunTests (Xunit.Abstractions.ITestFrameworkExecutor executor, System.Collections.Generic.IEnumerable`1[T] testCases, Xunit.IMessageSinkWithTypes executionMessageSink, Xunit.Abstractions.ITestFrameworkExecutionOptions executionOptions) [0x00008] in <4d712162c0a34cb99c967a7a106084df>:0
at Xunit.Runner.VisualStudio.VsTestRunner.RunTestsInAssembly (Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IRunContext runContext, Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IFrameworkHandle frameworkHandle, LoggerHelper logger, Xunit.Runner.VisualStudio.TestPlatformContext testPlatformContext, Xunit.Runner.VisualStudio.RunSettings runSettings, Xunit.IMessageSinkWithTypes reporterMessageHandler, Xunit.Runner.VisualStudio.AssemblyRunInfo runInfo) [0x00511] in <42d73e9a900048ada4679362440112c9>:0
No test is available in /home/user1/commandline28/commandline/tests/CommandLine.Tests/bin/Debug/net461/CommandLine.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Test Run Failed.
Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:.

Note: I checked this issue, but it is not relevant.
The solution in this issue is like my workaround solution.

Attach of linux session

[nohwnd]

I can see the same thing now. I would go with the workaround that you found. I couldn't think of any other way to solve this right now.

[moh-hassan]

What is the role of Microsoft.TestPlatform.ObjectModel in Linux?

[nohwnd]

That assembly holds all the common types. On windows it grabs it from the SDK where we ship testhost.exe, but on Linux that does not work for some reason.

[moh-hassan]

I installed xunit.console.exe v2.4.1 to run the test in Linux with Mono installed (Mono version 6.8.0.105).

Xunit test success
the next script is working:

mono path/to/xunit.console.exe   path/to/net461/test.dll       # net461 working

[moh-hassan]

@nohwnd
Also, If the package Microsoft.VisualStudio.TestPlatform.ObjectModel is added as a reference, test success.
The package Microsoft.VisualStudio.TestPlatform.ObjectModel is used by Xunit project and it's one of the dependencies of Microsoft.TestPlatform.ObjectModel.
It provides interfaces such as ITestExecutor and ITestDiscoverer for implementing custom unit test adapters.

It was requested by @bradwilson, the author of Xunit, here

The referenced assembly is a contract assembly which is expected to be provided by the version of Visual Studio (and/or VSTest) which is running the tests.

This package is used also by netcoreapp3.1 but it's loaded during netcoreapp3.1 build but it's not loaded during net461 build.
This part of .deps.json for netcoreapp3.1 project:

"Microsoft.TestPlatform.ObjectModel/16.6.1": {
        "dependencies": {
          "NuGet.Frameworks": "5.0.0"
        },
        "runtime": {
          "lib/netstandard2.0/Microsoft.TestPlatform.CoreUtilities.dll": {
            "assemblyVersion": "15.0.0.0",
            "fileVersion": "15.0.0.0"
          },
          "lib/netstandard2.0/Microsoft.TestPlatform.PlatformAbstractions.dll": {
            "assemblyVersion": "15.0.0.0",
            "fileVersion": "15.0.0.0"
          },
          "lib/netstandard2.0/Microsoft.VisualStudio.TestPlatform.ObjectModel.dll": {
            "assemblyVersion": "15.0.0.0",
            "fileVersion": "15.0.0.0"
          }
        },

I noticed that the package has no project reference and it's not maintained since 2017, but the package is included in netcoreapp3.1 as version 16.6.1.

Suggested solution
The package Microsoft.TestPlatform.ObjectModel should be part of the dependencies of the package Microsoft.NET.Test.Sdk in net4x and it can solve many like issues in Linux/Mac with Mono installed.