Vstest: [Windows ARM64] Unhandled exception when running dotnet test

[eiriktsarpalis]

Steps to reproduce

Using the latest arm64 .NET 5.0 preview SDK:

> .\dotnet-install.ps1 -Architecture arm64 -Version 5.0.100-preview.6.20318.15
> dotnet new xunit
> dotnet test

Expected behavior

Should successfully run the test included in the template.

Actual behavior

Throws an exception:

A total of 1 test files matched the specified pattern.
Failed to launch testhost with error: System.AggregateException: One or more errors occurred. (The specified executable is not a valid application for this OS platform.)
 ---> System.ComponentModel.Win32Exception (216): The specified executable is not a valid application for this OS platform.
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.ProcessHelper.LaunchProcess(String processPath, String arguments, String workingDirectory, IDictionary`2 envVariables, Action`2 errorCallback, Action`1 exitCallBack, Action`2 outputCallBack)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.LaunchHost(TestProcessStartInfo testHostStartInfo, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.<>c__DisplayClass38_0.<LaunchTestHostAsync>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__276_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.LaunchTestHostAsync(TestProcessStartInfo testHostStartInfo, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)

Test Run Aborted.
C:\Users\eitsarpa\AppData\Local\Microsoft\dotnet\sdk\5.0.100-preview.6.20318.15\Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [C:\Users\eitsarpa\devel\tests\msbuild-repro\msbuild-repro.csproj]

Diagnostic logs

logs.zip

Additional Info

Reproduces with the NUnit template as well. Works without issues using the x86 sdk.

Environment

vstest.console version: Microsoft (R) Test Execution Command Line Tool Version 16.7.0
OS info: Windows 10 build 18363 on Surface Pro X

[eiriktsarpalis]

Possibly related? https://github.com/microsoft/vstest/pull/1927 /cc @ViktorHofer

[nohwnd]

Seems to me that the test host provider does not take arm into account and tries to run testhost.exe that is built for x64 instead of running dotnet executable with testhost.dll as it would on Linux / MacOs.

TpTrace Error: 0 : 2752, 4, 2020/07/02, 18:34:18.114, 275749328108, vstest.console.dll, ProxyOperationManager: Failed to launch testhost :System.AggregateException: One or more errors occurred. (The specified executable is not a valid application for this OS platform.)

@eiriktsarpalis could you share a abit more about where you observe this error, to help us repro. We probably dont have ARM hardware on hand. Is it on a hosted ARM64 agent?

[eiriktsarpalis]

I'm reproducing it on my Surface Pro X laptop, which has an ARM64 chip.

[nohwnd]

Time to find the Raspberry Pi I have laying somewhere around I guess :D

[nohwnd]

My raspberry has v7 processor which is 32-bit. Trying to get some hardware to try this on.

[nohwnd]

Was able to install W10 on Raspberry Pi4 and the error indeed is that we fail to consider that W10 can run on ARM and try to invoke the testhost.exe that is built for Intel instead of falling back to dotnet+testhost.dll. Luckily, we recently added an internal option that allows dotnet/runtime team to provide their own dotnet executable. Using that option you can skip the faulty logic and force it to use dotnet+testhost.dll.

Example settings here. Just make sure you set the absolute path to an existing dotnet.exe, or the one that you built:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <RunConfiguration>
    <DotNetHostPath>D:\dotnet-sdk-5.0.100-preview.8.20359.11-win-arm64\dotnet.exe</DotNetHostPath>
  </RunConfiguration>
</RunSettings>

More info here:

https://github.com/microsoft/vstest/issues/1749#issuecomment-587375392

---

I will try to fix this, but it might take a while till it gets back into SDK.

---

I also saw these issues, that might be caused by the fact that I am building it on x64 with win10-arm64 RID, and then copying that on USB drive to the target system, my ethernet does not work on the RPi 😁 :

• no output after running dotnet test:
  Run it with -v d and you'll see that there is an error regarding IsTestProject not being true. Instead point dotnet test directly at the .dll that you want to test.

• runtime 2.1.0 not found.

We build against netcoreapp2.1, but I saw this when I tried running my own version of vstest.console.dll, because it defines that runtime in the runtimeconfig. I simply allowed any newer runtime to be used by using DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX=2

[eiriktsarpalis]

cc @ViktorHofer. We might be able to fix dotnet test for arm64 using the above. Or we could just wait until we get the fixed bits, your call.