Runtime: Enable .NET Core on Yocto Linux

[Tragetaschen]

In https://github.com/Tragetaschen/meta-aspnet/tree/master/recipes-dotnet/corehost, there are some bitbake recipes that compile CoreCLR, CoreFX and core-setup for inclusion a in Yocto Linux based distribution for ARM.

The git commits used follow the versions in pkg/projects/Microsoft.NETCore.App/project.json.template, to get to a consistent set of packages for core-setup to pick up from a local NuGet folder.

As a RID, this setup currently (mis)uses ubuntu.16.04-arm, but of course it's not Ubuntu. I haven't yet found a good way to get to a better RID since Yocto/OpenEmbedded is a meta-distribution used to create actual, embedded distributions depending on ones needs. I'll try to summarize my thoughts below.

The recipes currently contain a handful of patches that tame the build environment to actually compile. I'll create some issues in the appropriate repos.

And before I forget:
Many thanks to @ernstp for doing all the hard work enabling the 2.0 Yocto builds!
And of course to all the people that ported .NET Core to ARM in the first place!

[Tragetaschen]

Regarding a RID for Yocto/Linux, I'm currently trying to bring together the meta-distribution nature on the one hand and the wish to target the resulting distribution with my own nupkgs.

Yocto itself has the concept of releases (the current one is called morty or 2.2), but this only denotes a couple of "recipes" for libraries and programs that can be included in an actual distribution. For example, Yocto itself comes with a default distribution called poky (again version 2.2), and also a poky-tiny, a poky-lsb and a poky-bleeding one.

Beyond that, you are free to define your own distributions (mine is called target with version 1.0). Each distribution can be augmented by additional "layers" (e.g. "meta-aspnet", or "meta-mono") that can quite freely add additional, reconfigure existing, or override versions for packages. For example, "meta-aspnet" requires "meta-clang" so that compiler becomes available. Once you start customizing images beyond what was released as "Yocto 2.2", you are responsible for the integration testing on your actual system.

In the end, a yocto.2.2-arm RID (plus yocto.2.2, yocto, yocto-arm towards linux-arm) doesn't mean anything specific—a package author doesn't know what to expect—but its something known beyond a specialized Yocto distribution. On the other hand the "correct" RIDs like poky.2.2-arm or target.1.0-arm won't have a place for example here in the core-setup context, but are something that I would like to target in a package (non-published of course)

[JensNordenbro]

My vote for this too. All the other efforts for .net core on arm centers around the iot or consumer level of linux. This efforts is about embedded linux, where you create your own minimal distribution.

[JensNordenbro]

@Tragetaschen, would it be possible to do it differently? Define a meta-dotnet-core-dependencies that is binary compatible towards tizen/ubuntu/whateverdist most aligned to poky, "supported" by Microsoft, hence aligning to a rid and therefor being able to run daily builds from coresetup and consume nugets with native parts targeting all common rids?

[darrensteadman]

Has this progressed any further? I'd be very interested in being able to get .NET Core running on Yocto.

[frankhommers]

+1 same for me too. Would be great to write embedded in C#

[jamespettigrew]

Would like to see this too.

[pauldotknopf]

Can we get some input from the Microsoft team? I feel like we should have them weigh in on how the RID should work.

[Petermarcu]

My first reaction is that while rids may be able to be used for this, the complexity of trying to describe and represent all kinds of intersections will be close to a nightmare.

It is possible for someone to define an arbitrary RID graph. So based on a Yocto recipe that gets a name (like target from above) you could have a package you reference that brings in the RID definition for target which then would be used by NuGet to pick assets from the packages.

In summary, I dont think we should try to represent all the arbitrary subsets in the official RID graph but on a case by case basis, RIDs could be used within a scoped context.

@ericstj , can you do a small writeup on how people can add to the RID graph in their own package?

[Petermarcu]

I am hoping that the linux-arm RID works for Yocto assuming the recipe used pulls in at least the dependencies required by .NET Core. I need to put together a good list of dependencies that are required somewhere.

[Petermarcu]

Looks like in https://github.com/Tragetaschen/meta-aspnet/tree/master/recipes-dotnet/corehost we'll need to get core-setup building as well. Thats where the dotnet host builds and is the thing that knows how to act like .NET Core. Corehost is over simplified.

[JensNordenbro]

We run linux arm rid on yocto. We use tragetaschen to bring in dependencies and then use binaries linked from core setup Readme.md or just publish the source to self contained deployment using rid linux-arm. This way, as long as my yocto is glibc compatible, even third party nugets with native dependencies will work. The dotnet host in tragetaschen will also work but the last time I ran it, it assumed ubuntu-arm rid.

[Petermarcu]

Great, yes, thats what I hoped. So as long as you have the dependencies built into your yocto recipe, the binaries provided in our packages for linux-arm just work.

[pauldotknopf]

Out of curiosity, what would happen if I bring in Libuv as a dependency package? It doesn't have the linux-arm binaries. Does that mean a recipe will have to be built that will compile the libuv .so files?

[Petermarcu]

The latest preview package does have linux-arm. https://www.nuget.org/packages/Libuv/1.10.0-preview1-22040

[JensNordenbro]

It would be nice @Petermarcu if Microsoft at least provided a list for coreclr, corefx native dependencies or even better this list translated to a yocto definition...
Or maybe a cli extension to validate coreclr, corefx prerequisites. Like

dotnet validate-system

[Tragetaschen]

As far as I understand, .NET core has moved away from distro-specific RIDs and now only uses the generic ones, for example linux-x64 for all the Ubuntus, Fedoras, Suses. I've recently changed meta-aspnet to build for the linux-arm RID as well. The experience with the x64 distros has been that the native libraries that are required for .NET core all work without recompilation on every x64 Linux and this includes libuv.

I actually thought about using the official binary linux-arm packages for Yocto because following the changes in the build process has been quite a challenge so far.

@JensNordenbro I get that it would be nice, but that's actually quite hard: You might for example break .NET Core by removing certain kernel features. I guess it's the responsibility of the distribution (author) to validate that. For x64 and on a supported distribution it's Microsoft, other x64 distributions might do their own work to include .NET core and make sure it actually works. For Yocto, it's the responsibility of the recipe to properly declare necessary dependencies for example with .bbappend files in the layer. All this work is only required once and not per computer, so a validate-system call is probably overkill.

[pauldotknopf]

@Tragetaschen I notice some of your patches are specific to ARM processors. Do you plan on making the build scripts use whatever target toolchain is configured? x86_64?

[pauldotknopf]

@Tragetaschen I suggest maybe making the runtime id configurable, via a machine.conf file. Then, hard-code "linux-arm" or "linux-x86_64" in each machine configuration.

[JensNordenbro]

@Tragetaschen , Me personally, I am more into downloading official linux-arm builds from Microsoft rather than building it on my own, but it's good to know that your Yocto stuff now builds for linux-arm rid!
Also it is the notion of building Self Contained Deployments that make me think that Microsoft should at least provide a list of "userland"-dependencies to simplify these scenarios. By the way @Petermarcu , when do Microsoft decide to bundle a lib and when not to?

[nzain]

The yocto layer "meta-aspnet" from @Tragetaschen helped me kickstart a bare bones linux (angstrom morty) on armv7l hardfloat where a .net core 2.0 hello world app runs happily. The following two solutions worked:

1. Use the (yocto) bitbake recipes from his layer to compile & install coreclr, corefx, and corehost
2. Download the Release/2.0.X armhf build from https://github.com/dotnet/core-setup (v2.0 is becoming stable at the time of writing) and install manually

The key requirement for the second solution are the native dependencies (libunwind, libicu, and such) which are nicely pulled & installed by the meta-aspnet yocto layer. Apart from that, the latest build seems to work just fine. The important task for the dotnet team is to create a well-formed & maintained list of native dependencies. There are several sites about dependencies, mostly raspberry pi tutorials, but they are scattered.

Side notes: The c# project must be built using 2.0, too - running a core1.1 app did not work. Compiling the csproj with explicit target RID (dotnet publish -r linux-arm) works, too. However, the same native dependencies are required and I don't see the benefit over a generic framework-dependent deploy.

Today I could only test a hello world app, will work my way up to fully fleshed asp webapi tomorrow.
EDIT: asp.net core 2.0 preview works like a charm, including swashbuckle. Thats all I need on the arm device.

[fidergo-stephane-gourichon]

Hello everyone. This issue is still opened and no comment for a year. Does it mean that @Tragetaschen's meta-aspnet runs smoothly? :-) Or that no one cares any more? :-(
Also I see https://github.com/pauldotknopf/meta-dotnet/commits/master which seems closer to the current topic?

[Tragetaschen]

I'm not even close to losing interest, but my current work load doesn't allow for anything substantial :-/

[nicolasr75]

Great interest over here, we're just not yet ready with our custom hardware and the Yocto distro. We plan on delivering all our future measuring devices with a Yocto and an ASP.NET Core provided configuration interface and REST interface! Up to now we have only developped .NET Core command line apps or Avalonia backed GUI apps running on Raspberry PIs with Linux Mate / Ubuntu. All running without any major troubles!

[ghost]

I'm trying to get .NET Core 3.0 compiling under Yocto, but it isn't easy. I already got .NET Core 2.2 running with the meta-aspnet Layer from @Tragetaschen. But there were many changes from 2.2 to 3.0, so porting these recipes is difficult. Are there any efforts from Microsoft to support Yocto?