Runtime: Arch Linux: Build.sh Managed Is Broken With Wrong Paths

[vindicatorr]

$ ./build.sh x64 release
Unsupported Linux distribution 'arch' detected. Configuring as if for Ubuntu.
Setting up directories for build
Restoring NuGet.exe...
Unsupported Linux distribution 'arch' detected. Downloading ubuntu-x64 tools.
Installing dotnet cli...
Restoring BuildTools version 1.0.25-prerelease-00190...
Failed to initialize CoreCLR, HRESULT: 0x80131500
ERROR: Could not restore build tools correctly. See '<corefxPath>/init-tools.log' for more details.
Initializing BuildTools...
<corefxPath>/init-tools.sh: line 83: <corefxPath>/packages/Microsoft.DotNet.BuildTools/1.0.25-prerelease-00190/lib/init-tools.sh: No such file or directory
Done initializing tools.
./build.sh: line 108: <corefxPath>/Tools/corerun: No such file or directory

tail: cannot open '<corefxPath>/msbuild.log' for reading: No such file or directory
Build Exit Code = 127

"dotnet-ubuntu-x64.1.0.0.001504.tar.gz" does not contain ANY .sh
Files are extracted to "./Tools/dotnetcli/bin/", so corerun will not be found in ./Tools.

[stephentoub]

What OS and version are you building on?

[vindicatorr]

$ uname -a
Linux computerName 4.5.0-rc3-ARCH-dirty dotnet/corefx#1 SMP PREEMPT Wed Feb 10 18:47:11 CST 2016 x86_64 GNU/Linux

[stephentoub]

No explicit work has been done to enable Arch Linux. We'd welcome you submitting PRs to fix places where things are currently broken, but other than reviewing such PRs and helping to get them merged, we're not currently focused on enabling it. You may need to build all of the managed components on another OS (e.g. RedHat, Ubuntu, CentOS, OS X, Windows, etc.) and copy them over to your Arch Linux install, as there's no dotnet-cli or platform-specific packages built for it currently.

[vindicatorr]

I'll try it, but I don't see how the dist difference would change the fact that the script was looking for corerun in Tools when the .gz would be extracted into Tools/dotnetcli/bin.

[shahid-pk]

@vindicatorr actually the problem is here in corefx not in dotnet-cli even if you extract the pre built binaries it still won't work because it will pull packages that are not available for arch-linux. Their are some packages that depends on platform or distro specific native libraries and their specific versions like libcurl etc so you need to port those packages to arch-linux here. and then add arch-linux to dotnet-cli as supported.

[ghost]

@vindicatorr, here is the quick / rough recipe for porting dotnet core:

# Step 1 - Build CoreCLR (Debug)
git clone https://github.com/dotnet/coreclr ~/projects/coreclr; cd coreclr ; ./build.sh

# Step 2 - Build CoreFX Native Libraries (Debug)
git clone https://github.com/dotnet/corefx ~/projects/corefx ; cd coreclr ; ./build.sh native

# Step 3 [Option 1 - From source] - Build mscorlib - the primary 'dotnet assembly' (Debug)
# You can't on Arch, so you need to use Windows VM to build:
git clone https://github.com/dotnet/coreclr
coreclr/build.cmd linuxmscorlib # windows
# then scp onto Arch:
scp -r coreclr/bin/Product/Linux.x64.Debug/mscorlib.dll \
user@ip:~/projects/coreclr/bin/Product/Linux.x64.Debug/mscorlib.dll

# Step 3 [Option 2 - Grab mscorlib from latest CI artifacts] (Debug)
#
# 1. Click the debug badge of the closest distro from
#      https://github.com/dotnet/coreclr/blob/master/README.md#build-status
# 2. Click on 'Last successful artifacts' and grab the .zip URL
#     e.g. http://dotnet-ci.cloudapp.net/job/dotnet_coreclr/job/debug_debian8.2/lastSuccessfulBuild/artifact/
# 3. Extract to /tmp and then copy mscorlib.dll to
#     ~/projects/coreclr/bin/Product/Linux.x64.Debug/mscorlib.dll.

# Step 4: Build CoreFX' managed assemblies
# This needs to do on Windows, Ubuntu, Debian or any supported distro
corefx/build.cmd /p:OSGroup=Linux  # Windows
corefx/build.sh  # Ubuntu et al
scp -r corefx/bin user@ip:~/projects/corefx

# Step 5: Run CoreFX tests
# and finally run tests with already built managed assemblies (that were copied from Windows)
corefx/run-test.sh \
    --coreclr-bins           ../coreclr/bin/Product/Linux.x64.Debug/   \
    --mscorlib-bins          ../coreclr/bin/Product/Linux.x64.Debug/   \
    --corefx-tests            ./bin/tests/Linux.AnyCPU.Debug/          \
    --corefx-native-bins      ./bin/Linux.x64.Debug/Native/

If you stuck on any step, open the issue in either CoreFX or CoreCLR repos or go to corresponding gitter channel http://gitter.im/dotnet/core[fx/clr].

• Good Luck! :star:

Edit: Updated Step 3.1 (mscorlib can only build on Windows), added new Step 4 and old Step 4 is now Step 5

[tmds]

@vindicatorr There is already an error on the line before the one that starts with 'ERROR'. This error says the coreclr that was downloaded as part of the tools did not run on your system. It is likely this is the cause of the next error.
@stephentoub Documentation on the topic of porting to an unsupported flavor of linux would be a great help. When doing the port, at some point you need to run the core runtime and c# compiler on a system for which you are building support. I guess some sort of bootstrapping is needed here? A statically linked coreclr with necessary libraries to run MSBuild and csc perhaps? I don't think this is available.

[ghost]

@tmds, yes we need to build coreclr for each distro separately. The jack-of-all-trades kinda cross-distro-compatible Linux build is usually possible when you build against old glibc (like for example node.js is built on CentOS 5.11 for all distros including Alpine -- which makes musl-libc guys unhappy because of hard dependency on glibc which is disfavored in the said platform environment ..), but it doesn't work in case of CoreCLR because the dependencies are handpicked for each distro separately -- which is a right way to do it. See https://github.com/dotnet/corefx/issues/2302#issuecomment-156225627.

[tmds]

@jasonwilliams200OK indeed, using the platform-provided libraries is the preferable thing to do. To get the build kicked off on a new platform, don't you need some statically linked runtime bootstrap system to build the 'real' system (which uses the platform libs)? Or how is it done?
I would love to see some documentation explaining how to get a working dotnet core on Ubuntu 15.10 (or Arch linux or ...)

[ghost]

Generally, if you want help with porting dotnet core, you have the whole dotnet community at your disposal:
https://gitter.im/dotnet/coreclr

don't you need some statically linked runtime bootstrap system to build the 'real' system (which uses the platform libs)?

Sure I am looking forward to such a solution and I do believe it is not impossible to setup. However, I think there are two issues at this point:

• CoreCLR and CoreFX cmake dependency introspection would need to be reconfigured.
• Entire dotnet core has many moving parts and things are moving faster in terms of evolution/progress; maintainability of such transient configuration is trivial.

When we worked on FreeBSD port, we contemplated three parts: CoreCLR, CoreFX and BuildTools.

• Step by step 'Building CoreFX on FreeBSD 10 HyperV VM' -
  https://gist.github.com/jasonwilliams200OK/18d9de1f5c78ed513e47
• Step by step 'Building CoreCLR on FreeBSD 10 HyperV VM' -
  https://gist.github.com/jasonwilliams200OK/06db121ec534821fe541

Now that information is pretty much obsoleted as is; when we (recently) worked on NetBSD/Alpine port after couple of months, there is a forth piece to the mix, dotnet-cli, which only builds on selective Unices (see dotnet/cli#1532). Besides there have been couple of changes to the build scripts across these repos.

The steps I have mustered above constitute most recent method to unlock yourself and build dotnet core on the Linux distro of your choice. That information was gathered in batches from dotnet team (mainly @ellismg and @stephentoub). Yes it would be nice to document it, but again pretty soon the infrastructure may evolve (which is naturally for betterness..)

explaining how to get a working dotnet core on Ubuntu 15.10

For Ubuntu 15.10, there is a ready infrastructure through and through. You can get the readymade CLI setup from https://dotnet.github.io/, most recent build of CLR from CI: https://github.com/dotnet/coreclr#build-status or run core[clr/fx]/build.sh to build everything from source. CLI, CLR, CoreFX, BuiltTools, MSBuild; everything is Ubuntu ready at the moment.

For new Linux distros such as Arch, it is comparatively easier to port by following the aforementioned steps, get CoreFX assemblies tested and announce the good news. For other Unices, BSD-based and Solaris likes, it requires rather deliberate and thorough work on all tiers of CoreCLR and CoreFX.

[vindicatorr]

@jasonwilliams200OK bea-uti-fal. I went with your approach but used ubuntu 15.04 instead since I had it set up in a VM.
coreclr build.sh wants 14.04, but when I edited it to allow 15.04, it still built:

$ cp -v ./build.sh{,.bak}
$ sed -e "s/14\.04/15\.04/" ./build.sh.bak | tee build.sh

On the flip-side, the run-test.sh in corefx doesn't like something:

$ ./run-test.sh \
>     --coreclr-bins           ../coreclr/bin/Product/Linux.x64.Release/   \
>     --mscorlib-bins          ../coreclr/bin/Product/Linux.x64.Release/   \
>     --corefx-tests            ./bin/tests/Linux.AnyCPU.Release/          \
>     --corefx-native-bins      ./bin/Linux.x64.Release/Native/ 2>&1 | tee run-test.log
Test project file ./src/System.Diagnostics.TextWriterTraceListener/tests/System.Diagnostics.TextWriterTraceListener.Tests.csproj indicates this test is not supported on Linux, skipping
error: Did not find corresponding test dll for ./src/Microsoft.VisualBasic/tests/Microsoft.VisualBasic.Tests.csproj at ./bin/tests/Linux.AnyCPU.Release//Microsoft.VisualBasic.Tests/dnxcore50/Microsoft.VisualBasic.Tests.dll
error: Did not find corresponding test dll for ./src/System.Xml.XPath.XDocument/tests/System.Xml.XPath.XDocument.Tests.csproj at ./bin/tests/Linux.AnyCPU.Release//System.Xml.XPath.XDocument.Tests/dnxcore50/System.Xml.XPath.XDocument.Tests.dll
error: Did not find corresponding test dll for ./src/System.Console/tests/System.Console.Tests.csproj at ./bin/tests/Linux.AnyCPU.Release//System.Console.Tests/dnxcore50/System.Console.Tests.dll
...
error: Did not find corresponding test dll for ./src/System.Threading/tests/System.Threading.Tests.csproj at ./bin/tests/Linux.AnyCPU.Release//System.Threading.Tests/dnxcore50/System.Threading.Tests.dll
error: Did not find corresponding test dll for ./src/System.Threading.Timer/tests/System.Threading.Timer.Tests.csproj at ./bin/tests/Linux.AnyCPU.Release//System.Threading.Timer.Tests/dnxcore50/System.Threading.Timer.Tests.dll
143 test(s) failed

I did take a look at glitter.im. It looks a lot like slack. I went ahead and posted there as well.

[ghost]

@vindicatorr, you are using release build for both CoreCLR and CoreFX right? Have a look at the CoreFX CI is running the test on Ubuntu 15:

http://dotnet-ci.cloudapp.net/job/dotnet_corefx/job/ubuntu15.10_release_tst/336/console

./run-test.sh                                             \
--configurationGroup Release                              \
--os Linux                                                \
--corefx-tests ./bin/tests/Linux.AnyCPU.Release           \
--coreclr-bins ../coreclr/bin/Product/Linux.x64.Release/  \
--mscorlib-bins ../coreclr/bin/Product/Linux.x64.Release/

(note that CI is also showing warnings like Test project file ./src/System.IO.FileSystem.AccessControl/tests/System.IO.FileSystem.AccessControl.Tests.csproj indicates this test is not supported on Linux, skipping)

[vindicatorr]

@jasonwilliams200OK heh, I was looking at all of the console output at jenkins and was flummoxed why I was seeing:

...
20:55:42 [ 4:55:55.63] Building Native Libraries...
20:56:05 [ 4:56:23.34] Successfully built Native Libraries.
20:56:05 [ 4:56:23.34] Building Managed Libraries...
...

http://dotnet-ci.cloudapp.net/job/dotnet_corefx/job/ubuntu15.10_release_bld/349/consoleFull
when the steps you offered was explicitly omitting "managed".
So when I looked over your steps again, something wasn't meshing. I couldn't find the ubuntu step that I thought I had seen (thought maybe I made it up in my head). Then I saw other changes and finally your "edit" comment. :)

Was really hoping to avoid the Windows route. I remembered it from the docs last year when I looked at the project and thought maybe ya'll finally were able to move away from it.
But when I saw

20:55:11 d:\j\workspace\ubuntu15.10_r---72a2fb9c>call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86   && build.cmd /p:ConfigurationGroup=Release /p:OSGroup=Linux /p:SkipTests=true /p:TestNugetRuntimeId=ubuntu.14.04-x64

as well, it made me think otherwise.

Odd that the docs no longer mention building mscorlib in Windows, and that it will build in ubuntu (if I mess with things (or use 14.04, I assume)), but the tests completely fail (with 15.04).
I'll have to poke around some more and still need to try building mscorlib in Windows.

EDIT:
argh, just remembered this log was corefx, which makes it even more odd that the build is being done in Windows.

[ghost]

@vindicatorr, though we can always get latest mscorlib from the CI, I agree this is bit of an ordeal.

I remembered it from the docs last year

Note that the process is getting maturer by day. For instance last year (few weeks ago actually), to build CoreCLR, CoreFX and Roslyn, we had a dependency on Mono and PCL assemblies (and even worse; a specific version of Mono). That has been revoked from CoreCLR and CoreFX now (which matters because we _need_ to build part of these repos on target system) and it is WIP in case of Roslyn (which doesn't matter because Roslyn itself is a managed compiler package).

[stephentoub]

It looks like there are other issues tracking Arch Linux progress. I'm going to close this out for now. Feel free to re-open if necessary.