Sdk: dotnet run fails on ARM64

[richlander]

dotnet run doesn't work for me on Linux ARM64.

/cc @livarcocc @RussKeldorph

rich@live-pc:~/testapp$ uname -a
Linux live-pc 3.10.105-bsp-1.2 dotnet/sdk#4284 SMP PREEMPT Sat Oct 27 19:24:05 IST 2018 aarch64 aarch64 aarch64 GNU/Linux
rich@live-pc:~/testapp$ lsb_release -a
No LSB modules are available.
Distributor ID: neon
Description:    KDE neon User Edition 5.14
Release:        18.04
Codename:       bionic
rich@live-pc:~$ time dotnet new console -o testapp
The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on testapp/testapp.csproj...
  Restoring packages for /home/rich/testapp/testapp.csproj...
  Generating MSBuild file /home/rich/testapp/obj/testapp.csproj.nuget.g.props.
  Generating MSBuild file /home/rich/testapp/obj/testapp.csproj.nuget.g.targets.
  Restore completed in 2 sec for /home/rich/testapp/testapp.csproj.

Restore succeeded.


real    0m17.732s
user    0m23.360s
sys     0m1.910s
rich@live-pc:~$ cd testapp/
rich@live-pc:~/testapp$ time dotnet build
Microsoft (R) Build Engine version 15.9.8-preview+g0a5001fc4d for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 342.77 ms for /home/rich/testapp/testapp.csproj.
/usr/share/dotnet/sdk/3.0.100-alpha1-009697/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(142,5): message NETSDK1057: You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview [/home/rich/testapp/testapp.csproj]
  testapp -> /home/rich/testapp/bin/Debug/netcoreapp3.0/testapp.dll

Build succeeded.                                             
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:21.04

real    0m25.204s
user    0m25.080s
sys     0m2.180s
rich@live-pc:~/testapp$ time dotnet run
System.ComponentModel.Win32Exception (8): Exec format error
   at Interop.Sys.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setUser, UInt32 userId, UInt32 groupId, Int32& lpChildPid, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean shouldThrow) 
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.DotNet.Cli.Utils.Command.Execute()
   at Microsoft.DotNet.Tools.Run.RunCommand.Execute()
   at Microsoft.DotNet.Tools.Run.RunCommand.Run(String[] args)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

real    0m17.997s
user    0m23.190s
sys     0m2.150s
rich@live-pc:~/testapp$ time dotnet bin/Debug/netcoreapp3.0/testapp.dll
Hello World!

real    0m0.682s
user    0m0.570s
sys     0m0.060s
rich@live-pc:~/testapp$

[livarcocc]

This looks like a runtime error to me. Throwing an exception on Process.Start. cc @stephentoub

[RussKeldorph]

@BruceForstall Can you take a look?

@jashook

[richlander]

I'm wondering if "filename" points to an incorrect binary.

[RussKeldorph]

The current theory is that the CLI/SDK have packaged an incorrect (probably x64) binary at some point.

[richlander]

But it's still 64-bit! j/k

[BruceForstall]

Looks to me like the default build is linux-x64. If you use dotnet build -r linux-arm64, it works. Without this, it creates a ./bin/Debug/netcoreapp3.0/testapp which is an x64 executable. I presume the CLI invokes this as a process?

[livarcocc]

adding @peterhuene here as well. There is a chance this is an issue with the apphost we are using.

[peterhuene]

Could we check to see what NETCoreSdkRuntimeIdentifier is set to in Microsoft.NETCoreSdk.BundledVersions.props for the arm64 builds?

[peterhuene]

Also, could someone email me a binlog of the failing build?

[peterhuene]

Looking at the latest tar.gz for master, I see this in bundled versions:

<NETCoreSdkRuntimeIdentifier>ubuntu.16.04-x64</NETCoreSdkRuntimeIdentifier>

This is generated from $(CoreSetupRid). Should this not be linux-arm64 or are we using the wrong build property when generating the props file?

[MichaelSimons]

FYI - as I noted in dotnet/cli#10295 this is affecting ARM32 as well.

[livarcocc]

We are using the wrong property. That explains why someone reported that dotnet --info for linux-arm64 was displaying ubuntu as the runtime, instead of linux-arm64.

[peterhuene]

To clarify: using wrong property when setting NETCoreSdkRuntimeIdentifier or does CoreSetupRid have the wrong value for arm64 builds? I assume the latter since NETCoreSdkRuntimeIdentifier doesn't affect dotnet --info.

[livarcocc]

We produce arm builds by cross-compiling. We build from ubuntu to linux-arm/arm64 by passing the Architecture. We change some properties in there when doing that.

I think it is very possible that we are using the wrong property (CoreSetupRid) on some parts of build.

cc @johnbeisner who made the linux-arm changes.

[peterhuene]

It seems like SharedFrameworkRid would be more appropriate to use for setting the SDK RID, as I see a an explicit override based on Architecture when UsePortableLinuxSharedFramework is set (I assume this is how we get the correct shared frameworks for arm builds currently?).

Seems odd that CoreSetupRid is effectively an alias of HostRid even when doing cross-builds, but I can't say I understand our packaging logic very well.

Is using SharedFrameworkRid what should be done here or should I make the setting of NETCoreSdkRuntimeIdentifier conditional on UsePortableLinuxSharedFramework/Architecture which we seem to have explicit checks for in a number of places?

[richlander]

@peterhuene the build isn't failing, just run. Will a binlog help with that?

[peterhuene]

I have one now, thanks!

The problem is that dotnet build is using NETCoreSdkRuntimeIdentifier to determine which apphost to use by default, which was incorrectly set to the host RID when building the .NET Core SDK for arm64 (thus ubuntu.16.04-x64 instead of linux-arm64 due to cross-building). dotnet run sees that an apphost was created by the build, so it attempts to exec it, resulting in the format error since this is x86-64 and not arm64.

This is a packaging issue for the .NET Core SDK, which is where NETCoreSdkRuntimeIdentifier is being set.

I'll try to sync up with John and get the fix into dotnet/core-sdk.

[richlander]

Thanks!

[richlander]

In case anyone is interested, this is the rig I was testing on: https://www.pine64.org/?page_id=3707

My latest toy.

[peterhuene]

Looking at the setup code again, I suspect ProductMonikerRid might be the correct property to use for NETCoreSdkRuntimeIdentifier.

I'll make a fix for that unless someone better versed in dotnet/core-sdk (@livarcocc @johnbeisner) have any objections.

[peterhuene]

@richlander by the way, what does your dotnet --info report for the build you're using?

[BruceForstall]

fyi, mine:

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview-009736
 Commit:    d273cdc5af

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  16.04
 OS Platform: Linux
 RID:         ubuntu.16.04-arm64
 Base Path:   /home/robox/brucefo/2/sdk/3.0.100-preview-009736/

Host (useful for support):
  Version: 3.0.0-preview-27106-02
  Commit:  410cba8501

.NET Core SDKs installed:
  3.0.100-preview-009736 [/home/robox/brucefo/2/sdk]

.NET Core runtimes installed:
  Microsoft.NETCore.App 3.0.0-preview-27106-02 [/home/robox/brucefo/2/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[peterhuene]

This should now be fixed in 3.0.100-preview-009738. I don't have an ARM device at the ready to test, but I did verify that the RID was correct in the bundled versions props file for arm tar.gz.

[richlander]

I validated that this bug is now fixed, with 3.0.100-preview-009762.

[texasbruce]

Still failing on .net 5:

[root12:55myAop]$ ../dotnet-sdk/dotnet --version
5.0.101                                                    
[root12:55myAop]$ uname -a
Linux localhost 4.14.116 #1 SMP PREEMPT Fri Oct 23 15:35:03 CST 2020 aarch64 GNU/Linux                                
[root12:55myAop]$ ../dotnet-sdk/dotnet build               
[root12:56myAop]$ ../dotnet-sdk/dotnet run                                                                            
The build failed. Fix the build errors and run again.      
[root12:56myAop]$ ls                                       
myAop.csproj  Program.cs