Runtime: Add support for running dotnet core on Banana linux

[tmds]

There are some pull requests on supporting specific well known distributions in this repository. I created this issue to improve documentation (and perhaps tooling) for the general issue.

Banana Unix is the term used by @jasonwilliams200OK in https://github.com/dotnet/cli/issues/1532 for some platform that doesn't support coreclr yet. This could be the next version of Debian or it may just as well be a custom built linux system for an embedded device.

Some concrete questions:
What are the minimal parts that need to be built for supporting a new platform? In what repos are they? And with which commands can they be built?
Can these bits be built on an existing supported platform? Or does the build have to be done on the new platform?
If the latter, is there any bootstrapping needed (e.g. get a first-stage core runtime to build the one for the new platform)? How is this done?

\cc @jasonwilliams200OK @ellismg @stephentoub

[ghost]

Banana Unix is the term used by @jasonwilliams200OK in dotnet/cli#1532 for some platform that doesn't support coreclr yet.

'Banana assembly' term was coined by @davidfowl in dotnet/corefx#4333 (now in Documentation/architecture/net-platform-standard.md#guard-rails-supports). I just took 'inspiration' from the better artist. :smile:

[joshfree]

Documenting the process of running on or porting to a new arbitrary distro is definitely on our radar. We're focused on RC2 / CLI right now but we do want to get to this general issue in the coming months.
/cc @blackdwarf

[sec]

Any update on this? What's needed to get dotnet build from sources?

[ghost]

@sec, if CoreCLR and CoreFX build on a platform, then its pretty much upto the dotnet team to bridge the steps till the platform package manager. The best we can do at this point is to port CoreCLR and CoreFX and make sure all the native+managed test suites (~4GB of tests) pass. This sometimes need quite a figuring out part.

After that we can just wait for packaging and muxer bootstraping etc. to happen by dotnet team.

[tmds]

@jasonwilliams200OK it would be nice if you could do the whole thing. E.g. crosscompile for some flavor of linux and chosing an rid for that. For example the Raspbian guys could make raspbian dotnet.

[ghost]

@tmds, see https://github.com/dotnet/coreclr/issues/917. This discussion is going on at least at 5 different places. Would be nice to combine all that into one thread.

[sec]

@jasonwilliams200OK lets assume I have coreclr and corefx compiled, tests passed - what's the next step to build dotnet cli binary on unsupported (FreeBSD in this example) OS ? I cannot find any info anywhere - build.sh keeps downloading bianries for ubuntu etc ?

[ghost]

@sec, this is the exactly kind of conversation I had and it is a complex process: https://github.com/dotnet/coreclr/issues/917. I will try it for Alpine aports and see how it goes. If/when i succeed I will write a step-by-step guide and send a PR to https://github.com/dotnet/core-docs. DIY FTW! :sunglasses:

[sec]

@jasonwilliams200OK Yes I've read that also :) I would like to do some hacking myself in the meantime - if you could just point me to proper repo/script to start building dotnet cli - maybe I can get it somewhere.

[sec]

[dotnet@freebsd ~/core-setup]$ ./cli/exe/dotnet --version

Microsoft .NET Core Shared Framework Host

Version : 1.0.2-beta-000555-00
Build : 42de2408f4707f0ae9bba5c2e5470a3ede284f27

The next is to build Microsoft.NETCore.App - any lead for this? :)

The specified framework 'Microsoft.NETCore.App', version '1.0.0' was not found.

• Check application dependencies and target a framework version installed at:
  /proc/2725/shared/Microsoft.NETCore.App
• Alternatively, install the framework version '1.0.0'.

[obadz]

I'm also stuck trying to build corefx on Nix (I've got coreclr built: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/coreclr/default.nix)

It seems that the build script attempts to download a tarball (something like https://dotnetcli.blob.core.windows.net/dotnet/preview/Binaries/1.0.0-preview2-002733/dotnet-dev-ubuntu.14.04-x64.1.0.0-preview2-002733.tar.gz) but it seems to also need MSBuild.exe which is not in that tarball.

Could you provide the list of inputs required for bootstrapping along with where we should place them?

(It should be assumed that package managers from most distros will not let the build process access the network to do its own downloads btw as that would obviously kill reproducibility/caching)

[janvorli]

@obadz bootstrapping new OS is now a very tedious proces. The tarball downloaded by the script is just a bootstrap tarball that enables it to actually get msbuild and other managed stuff to enable building the managed parts (said in a simplistic manner).
You can take a look at the end of the https://github.com/dotnet/coreclr/issues/917 where a discussion on adding Alpine support in a similar manner is going on.
cc: @mellinoe, @Petermarcu

[obadz]

hi @janvorli, is your last statement still true or are we now able to build the .NET tools from source+bootstrap binaries?

thanks

[janvorli]

@obadz it is still true. I am actually working on finally bringing up Alpine Linux support and I am recording the steps I needed to do so that I can write down and publish instructions. Hopefully sometime next week.
But as for the NixOS, if it uses glibc as the standard C library, the current dotnet 2.0 tarball should just work for you. The new version is portable, that means that it can be used on majority of Linux distros with glibc >= 2.12. I have tested Fedora, CentOS, RHEL, ArchLinux, Debian, Ubuntu, Mandriva, OpenSUSE, Gentoo myself.

[obadz]

Ah, great, thank you. Yes NixOS has glibc, but I'm interested in building from source.

[janvorli]

I think that even the build from sources should work for you now. We use the portable package as the tarball that the build downloads, so it should work.

[obadz]

@janvorli, that sounds good. Looking forward to read your instructions when you publish them. Cheers.

[obadz]

@janvorli,

Thought maybe I'd write a few questions to give you some inspiration for your instructions. What I seek, as a packager:

• Where do I download the source from (ideally also where can I find properly signed hashes for all the inputs)
• What dependencies does the project require to compile (libc, zlib, other sub projects, etc.)
• What architectures/OS does the project build & run on and any other considerations to arch/OS…
• How do I tell the project where it can find its dependencies—whether it is via an environment variable, a command line argument, an edit to a config file.
  
  
  
  • Projects should never impose a path for where it expects its dependencies to live, neither should it expect that several dependencies are in the same place.
  
  
• How can I tell the project where I'm going to want its output to be installed, for instance "--prefix=…"
  
  
  
  • Projects should never impose a specific path for that either, nor should they require that their output sit in the same directory as some other project
  
  
• What are the key compile time configuration flags (if any)
• In this particular case, where should I get the bootstrapping compiler, where should I place it in the build tree, and how do I make sure that none of the pre-built binaries end up in the final output
• How do I build the project, for instance "make"
  
  
  
  • These commands should never attempt to access the network, for any reason. Otherwise builds cannot be reproducible.
  
  
• How do I copy the relevant output of the project to the relevant folder, for instance "make install", or some other script, or sometimes even some manual copy things to the output file tree.
• How can I make sure the output is working properly (tests? run some command and expect some behavior? etc.)
• What run time configuration files does the project expect to find, and how can I tell it (at run time, or build time) where it should expect to find said file
  
  
  
  • Again, project should impose a path for that either (i.e. not /etc)
  
  

Thanks!

(EDITED once to add a couple of missing items)

[janvorli]

@obadz the instructions that I am working on are targeting a different audience. They cover the scenario when someone wants to add support for a new target OS / distro incompatible with the binaries we provide. Like Alpine Linux, FreeBDS, NetBSD etc. On such platforms, first a bootstrap command line tools need to be created so that there is something that can build managed parts of coreclr and corefx. Then the new RID has to be created and plumbed at various places. And finally the nuget packages with the coreclr and corefx components need to be built.

But obviously your scenario is also a very valid one. Besides providing some tarballs and rpm / deb packages with dotnet core, we would also love to see various distros out there having dotnet available as their native packages in their repositories.
If you look at https://github.com/dotnet/core-setup/tree/master/src/pkg/packaging, you can get some idea on how we build the deb and rpm packages. But ultimately, I guess the full end to end build from sources that's in the https://github.com/dotnet/source-build repo is the way for you to go. I am just familiarizing myself with it since I was not involved in its creation, so I cannot provide you with much info on how it works. As far as I know, the only precompiled prerequisite it needs is a dotnet cli package to bootstrap the build. But it should not include that in the output of the build.

[sec]

@janvorli Hey - did you have any draft published somewhere maybe? I noticed a lot changed since 1.x when it comes to building .net core on unsupported platform(FreeBSD in my case) ?

[janvorli]

@sec are you interested in helping to make the whole .NET Core build on FreeBSD? I've just given it a try today, coreclr native binaries build ok, corefx native binaries on the contrary don't build due to some differences in networking interfaces and gssapi.
Before attempting to build the bootstrap cli, we need to have both corefx and coreclr native parts building.

[sec]

@janvorli Yes I would like to help (what's in my power and time). I don't know much about cmake, but I've managed to compile CoreFX native on FreeBSD (I used release/2.0.0 branch). I'm interested in running .net core apps on bsd (develop/compile I can do on supported OS).

Command I used:
src/Native/build-native.sh freebsd x64 debug

Here are the changes I done to make it work - https://gist.github.com/sec/d11f2515603a824b85d8be5cd14daa7f

For the gssapi, 'pkg install krb5' will solve that problem.

The managed part of CoreFX can be compiled on Windows, this is still right?
What should be next step? I tried to build "cli" but it changed since 1.x release and can't find any updated document on how to "do this". What's the next step/repo that need's to be build?

EDIT:
only one change in code to make it compile, then run:
src/Native/build-native.sh freebsd x64 debug cmakeargs -DHAVE_IN_PKTINFO=0

Should I create PR for that code change (already have fork and commit done, but it's just to move one line around)

Don't know why test for HAVE_IN_PKTINFO have success code - maybe missing include somewhere

EDIT 2:
So I have coreclr and corefx compiled native and managed part.
Currently this is blocking me from doing further - https://github.com/dotnet/core-setup/issues/3131

EDIT 3:
Thanks to @janvorli, core-setup now also compiles. I was also able to run (using corerun) basic .NET Core 2.0.0 ASP.NET application using EF+SQLite (missing part was FileWatcher, for which I placed empty stub). So now I think it should be almost possible to create SDK package for FreeBSD (with missing DLL's like FileWatcher) - do we have any instruction hwo to do it? For example dotnet cli is looking for fxr under -1.-1.-1 directory which is version I assume.

[karelz]

@sec FYI: We are resurrecting FreeBSD port effort (tracking in https://github.com/dotnet/corefx/issues/1626#issuecomment-329840518). If you're interested, please join us there ...

[tmds]

I don't think there are any more actions for this in corefx.

I've created an issue in source-build to have a CI build that runs on an unknown RID: https://github.com/dotnet/source-build/issues/297.