Nixpkgs: Fix CoreCLR

[copumpkin]

Issue description

CoreCLR is currently marked as broken, because it's tricky to get building. @obadz helpfully posted a comment with issues encountered, which I'm just going to reproduce here. cc @ForNeVeR who's expressed interest in resurrecting CoreCLR too.

From @obadz:

My recollection is quite hazy on the details but basically it was never clear (or documented in a way that I could find) where I was expected to fetch a bootstrapping compiler, where I was expected to put it, what commands I was expected to invoke to run the build, and even: after the the build, what was the relevant output files to keep and how should they be organized in a certain directory structure to be usable.

coreclr is only one of the subprojects that one needs to build to get to a useful setup. There's also corefx and a few others. Was also not clear to me what the exhaustive list was.

Here are some relevant issues/pulls:

dotnet/corefx#6770 (comment)
dotnet/coreclr#917
dotnet/coreclr#1198
dotnet/coreclr#7573
dotnet/coreclr#7675

(see https://github.com/NixOS/nixpkgs/commit/1dd16a93741c425880ac81d1e611af7be050c96b#commitcomment-21961855 for full context)

[baracoder]

There are some prebuild packages. Would it be easier to use them instead? https://www.microsoft.com/net/download/linux

[copumpkin]

Probably easier, but we generally try to build from source where possible. But if it's a choice between not working at all or using prebuilt packages, I'll take the latter 😄

[obadz]

There seems to be some useful information here: https://github.com/dotnet/source-build (thanks @janvorli for pointing me to it here: https://github.com/dotnet/corefx/issues/6770#issuecomment-324491220).

I might give a go at building .NET Core on Nix again. Let's centralize communication on this topic on this issue.

[obadz]

I gave up without much of an attempt (sorry). Focusing my dev efforts on nimlang instead.

[kuznero]

I would like to jump on this and help. But I am trying to make an attempt upgrading it in the same time (coreclr 2.0.0). Currently it fails on build step complaining about glibc header:

...
In file included from /tmp/nix-build-coreclr-2.0.0.drv-0/coreclr-v2.0.0-src/src/pal/src/cruntime/file.cpp:22:
In file included from /tmp/nix-build-coreclr-2.0.0.drv-0/coreclr-v2.0.0-src/src/pal/src/include/pal/palinternal.h:578:
In file included from /nix/store/g65l985579ri87i8m3906a2nmyrl24a2-gcc-6.4.0/include/c++/6.4.0/stdlib.h:30:
/nix/store/zsfnlmvw8k7akg1s0ypdy5jd1kq0d8hn-glibc-2.25-49-dev/include/stdlib.h:63:5: error: typedef redefinition with different types ('struct DUMMY_div_t' vs 'struct DUMMY_div_t')
  } div_t;
    ^
/tmp/nix-build-coreclr-2.0.0.drv-0/coreclr-v2.0.0-src/src/pal/src/include/pal/palinternal.h:163:15: note: expanded from macro 'div_t'
#define div_t DUMMY_div_t
              ^
/tmp/nix-build-coreclr-2.0.0.drv-0/coreclr-v2.0.0-src/src/pal/inc/pal.h:5171:3: note: previous definition is here
} div_t;
  ^
/tmp/nix-build-coreclr-2.0.0.drv-0/coreclr-v2.0.0-src/src/pal/src/include/pal/palinternal.h:163:15: note: expanded from macro 'div_t'
#define div_t DUMMY_div_t
              ^
1 error generated.
make[2]: *** [src/pal/src/CMakeFiles/coreclrpal.dir/build.make:63: src/pal/src/CMakeFiles/coreclrpal.dir/cruntime/file.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:281: src/pal/src/CMakeFiles/coreclrpal.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
...

I can see that there is an assumption for coreclr to be compatible with glibc coming from ubuntu 14.04 which is 2.19 whether in nix it is 2.25. @copumpkin @baracoder @obadz Is that the issue that others observe as well or am I just doing something wrong?

[janvorli]

@kuznero this is strange, the mechanism in pal.h / palinternal.h with #define div_t DUMMY_div_t / #undef is supposed to prevent this kind of issue from happening.
You have mentioned 2.0.0 - does it mean you have checked out sources at release/2.0.0 branch?

[kuznero]

@janvorli I think it checks out archive released by 2.0.0 tag. So, I don't think it is a branch. Basically I just changed the version and a hash.

[janvorli]

@kuznero I have thought you are just trying to build coreclr repo by itself, but now looking back at the discussion, I guess you are trying to use the source_build repo, right? If that's the case, I would recommend to wait a bit since there is now ongoing effort to remove patches that the build from sources applies to various repos it builds and make it all sustainable w.r.t. changes in the repos it includes in the build. You can also help with this effort if you want. @weshaggard and @eerhardt are driving that.

[obadz]

I had to add these things to build this way back when. Maybe they are now causing the issue?

https://github.com/NixOS/nixpkgs/blob/1dd16a93741c425880ac81d1e611af7be050c96b/pkgs/development/compilers/coreclr/default.nix#L50-L63

[kuznero]

@obadz thanks, will try to play with it

[kuznero]

@obadz That seem to help. So, it just works. Do you have a chance to try it as well? On https://github.com/kuznero/nixpkgs/tree/pr/coreclr2 repo: nix-env -f . -i coreclr.

[ForNeVeR]

I've checked if the coreclr builds; it looks okay. But does anyone have an idea what we'll do next? CoreCLR is only a runtime part of the .NET Core SDK, right? Does that mean we still need to build dotnet-cli? Something else?

[kuznero]

In order to elaborate a bit more here, these are the repositories to package in order:

1. coreclr - hopefully done with the #30156
2. corefx
3. cli
4. roslyn potentially?

[simendsjo]

I'm interested in getting dotnet running on nixos, but I don't even have a working knowledge of nix (I'm cargo-culting everything). Anything I can do to help?

[obadz]

coreclr broken somewhere in 040bde90521c9956e8970b35305a337732cf0ade..ce83cd7ca324b8defd9165fe54704b201aabe6cb

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[leo60228]

This bug is fixed. However, a large part of this issue seems to be building .NET Core from source, which is still not done. Perhaps that should be a separate issue?

[janvorli]

There is a dotnet/source-build repo that is the intended way to build .NET Core from sources in various Linux repos. It seems it would make sense to try that.

[leo60228]

I tried that a while back. If I recall correctly, there was no clear way to
do the build without network access.

On Mon, Nov 16, 2020, 4:56 AM Jan Vorlicek [e-mail] wrote:

There is a dotnet/source-build repo that is the intended way to build .NET
Core from sources in various Linux repos. It seems it would make sense to
try that.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/25498#issuecomment-727869861,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB7X32NEQ2EITONUALD35HTSQDZLNANCNFSM4DKCQKSQ
.

[leo60228]

https://github.com/dotnet/source-build/issues/1887

On Mon, Nov 16, 2020, 7:28 AM leo60228 [e-mail] wrote:

I tried that a while back. If I recall correctly, there was no clear way
to do the build without network access.

On Mon, Nov 16, 2020, 4:56 AM Jan Vorlicek [e-mail]
wrote:

There is a dotnet/source-build repo that is the intended way to build
.NET Core from sources in various Linux repos. It seems it would make sense
to try that.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/25498#issuecomment-727869861,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB7X32NEQ2EITONUALD35HTSQDZLNANCNFSM4DKCQKSQ
.

[janvorli]

As far as I know, it should work without network access. I believe the build in source-build repo would produce a source tarball and that's what you'd use for the build on your NixOS.

[leo60228]

Yeah, I think I was mixing it up with whatever the predecessor to
source-build was. Do you know if pregenerated source tarballs are available
anywhere?

On Mon, Nov 16, 2020, 7:44 AM Jan Vorlicek [e-mail] wrote:

As far as I know, it should work without network access. I believe the
build in source-build repo would produce a source tarball and that's what
you'd use for the build on your NixOS.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/25498#issuecomment-727955286,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB7X32KGCOG4U5DKT6ZNT7TSQENCHANCNFSM4DKCQKSQ
.

[janvorli]

I don't know, it would be better to ask in the dotnet-source repo. But since the tarball already contains RID of the target platform, I guess it would not help you. And since NiXOS RID is not supported in .NET Core, it seems you'd need to add it first - either as a patch in the source-build (there is a support for automatically applying patches before build) or in the runtime and installer repos.
But I am no expert on source-build, so it would be preferable to get more details from folks working on the repo itself.