Nixpkgs: Problem Installing GHC on Raspberry Pi 3

[kalhauge]

Issue description

GHC does not install out of the box on raspberry pi 3 using the 'aarch64' platform.

Steps to reproduce

Install nixos on a raspberry pi using the description on the nixos, using the 'aarch64' version. And
upgrade to the newest version.

Set config.nix:

[kalhauge@nixos:~]$ cat ~/.config/nixpkgs/config.nix
{ allowBroken = true; }

Then try to install:

[kalhauge@nixos:~]$ nix-env -i ghc
installing ‘ghc-8.0.2’
error: cannot bootstrap GHC on this platform
(use ‘--show-trace’ to show detailed location information)

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

[kalhauge@nixos:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-linux"`
 - host os: `Linux 4.13.11, NixOS, 18.03pre119873.292f60504c (Impala)`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 1.11.15`
 - channels(root): `"nixos-18.03pre119873.292f60504c"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`

[kalhauge]

I was told to tag @peti on the IRC channel :)

[peti]

I am sorry, but we cannot bootstrap ghc for ARM yet, and I have no experience doing that kind of thing either and won't be able to help.

[kalhauge]

Thanks for the quick response, I will take a crack at it. I just wanted to make sure that it was not a solved problem.

[copumpkin]

@dezgeg and @Mic92 are also experienced with ARM-flavored things, I think

[dezgeg]

Upstream does seem to provide armv7 and aarch64 binary releases for 8.2.1: https://downloads.haskell.org/~ghc/8.2.1/. So I'd assume you'd need to create a pkgs/development/compilers/ghc/8.2.1-binary.nix that is used to bootstrap the rest of the compilers...

[dhess]

@kalhauge Put the files in this gist in your nixpkgs overlays directory: https://gist.github.com/dhess/f81be9858aef3563cf521f099ba65ab8

You'll also need to copy/symlink the patch files referred to in 8.0.2.nix from nixpkgs into your overlays directory.

The 8.0.1-binary.nix derivation in that gist will download the Debian packages for armv7l-linux and aarch64-linux and create a binary GHC from those. Then the 8.0.2.nix derivation uses that to bootstrap.

It's a bit hacky, but it does the trick. I've been building GHC for armv7l-linux and aarch64-linux this way for months.

The bootstrap process will probably take ages on an RPi3, so be prepared to wait :) Also make sure you have plenty of disk space.

Limitations/issues:

• It bootstraps from GHC 8.0.1 because I wrote this many months ago when that was the latest.

• Only works for derivations that use haskellPackages because that is what this overlay overrides. If you try to use haskell.packages.ghc821 or even ghc802, I believe you will still get that error. If I could figure out how to override the ghc802 attributes in haskell.compiler and haskell.packages, then it would probably work for anything that uses ghc802 to bootstrap. edit I think I've fixed this now, as well. I'm not sure I've done the haskell.compiler and haskell.packages overriding correctly in the haskell.nix overlay, but it seems to work for me, anyway. There should now be a haskell.compiler.ghc801Binary derivation, and the haskell.packages.ghc802 derivation (and anything that uses it, such as haskell.packages.ghc821) should now work on ARM systems, as well.

• As you can see, it currently ignores any overrides you might have set on haskellPackages when you build for an ARM platform. This could probably be hacked to use supers overrides, but I don't need it as I'm using overlays with extend now. I'll leave this as an exercise for the reader :) edit I think it should now respect haskellPackagesOverrides.

I only got this working as an overlay just yesterday -- previously, I was maintaining it as patches/my own fork against nixpkgs -- so I will continue improving it to make it more seamless, and will try to keep the gist updated as I go.

[kalhauge]

@dezgeg I was trying something similar yesterday, but I didn't get very far (https://github.com/NixOS/nixpkgs/compare/master...kalhauge:ghc-on-arm)

@dhess, Great work! Is there a reason you have not opened a pull request?

[dhess]

@kalhauge There are some reasons why I think this isn't ready for a pull request in its current form, and I don't know how to address them. I outlined the problems with the current approach here: https://github.com/NixOS/nixpkgs/issues/19926#issuecomment-302794606, when I replied to an issue about needing more GHC binary derivations for faster GHC bootstrapping with an earlier version of this work. Most of the problems I outlined in that comment still apply to this overlay version.

If someone can tell me how to fix/mitigate these limitations to ensure the work gets upstreamed, I'm happy to submit a PR, but I don't want to submit a PR in the current form because I expect that it would be rejected as-is.

[kalhauge]

It seems like there is already a PR on this https://github.com/NixOS/nixpkgs/pull/29688, and it also includes bootstrapping for aarch64.

@dhess, I tried your Gist, and it seems to work. You have to put the 8.0.2.nix and 8.0.1-binary.nix in another folder, as putting them directly in the overlays folder makes nix load them. This results in weird errors. I am currently confirming that all works by compiling everything, but it takes some time.

[dhess]

@kalhauge Yes, sorry. In my overlays tree, those files are in a pkgs/haskell subdirectory, but I was trying to make it a bit more generic and forgot that Nix would try to load all the .nix files in the overlays directory. Sorry about that.

[kalhauge]

@dhess the build finally completed but it failed:

...
[36 of 95] Compiling Distribution.PackageDescription ( libraries/Cabal/Cabal/Distribution/PackageDescription.hs, bootstrapping/Distribution/PackageDescr$ption.o )
ghc/ghc.mk:111: ghc/stage1/package-data.mk: No such file or directory
make[1]: *** [utils/ghc-cabal/ghc.mk:48: utils/ghc-cabal/dist/build/tmp/ghc-cabal] Killed
make: *** [Makefile:130: all] Error 2
builder for ‘/nix/store/mhxf77q9mvh7ba1rsyjmnhjbhdigsmj2-ghc-8.0.2.drv’ failed with exit code 2

Any Idea why?

[dhess]

@kalhauge No, but it's been awhile since I've built this on aarch64 (typically I only use the armv7l port). I will test it myself and get back to you.

[dhess]

Sorry, still haven't had time to revisit this yet, but I will try again in the next few days.

[kalhauge]

Thanks for still looking into it. In the meantime, I tried out #29688, but it too had problems.

[kalhauge@nixos:~]$ nix-env -f nixpkgs -iA haskell.compiler.ghc821Binary
...
"utils/ghc-cabal/dist-install/build/tmp/ghc-cabal-bindist" copy libraries/ghc-prim dist-install "strip" '' '/nix/store/9bzkjqzrncn1shv1z0a80ab2h58d94pz-ghc-8.2.1-binary' '/nix/store/9bzkjqzrncn1shv1z0a80ab2h58d94pz-ghc-8.2.1-binary/lib/ghc-8.2.1' '/nix/store/9bzkjqzrncn1shv1z0a80ab2h58d94pz-ghc-8.2.1-binary/share/doc/ghc-8.2.1/html/libraries' 'v p dyn'
Saved package config file is corrupt. Re-run the 'configure' command.make[1]: *** [ghc.mk:991: install_packages] Error 1
make: *** [Makefile:51: install] Error 2
builder for ‘/nix/store/6bkhk5ngklp7zfgz5pqwlwgk7n8rigf7-ghc-8.2.1-binary.drv’ failed with exit code
2
error: build of ‘/nix/store/6bkhk5ngklp7zfgz5pqwlwgk7n8rigf7-ghc-8.2.1-binary.drv’ failed

[dhess]

@kalhauge Finally got around to looking into your issue. I'm building GHC 8.0.2 now for aarch64-linux. It's taking awhile, but I'll update this comment later when the build has finished.

edit 1 It ran out of memory, but got well past the point where you had issues. Unfortunately, I'll have to up the swap and start over again, but I think it looks promising.

edit 2 It finally built! It took about 12 hours and I had to bump the swap up to 4GB on my Jetson TX1 with 4GB RAM. Here is the output of ghc --info: https://gist.github.com/dhess/f612eef069d6c6463fae9ca7e12c1a73

Also, I've now published this overlay as part of https://github.com/quixoftic/quixops-modules. If you clone that repo somewhere and then point your overlays to the quixops-modules/overlays/haskell.nix overlay, you can build the exact same derivations I'm building.

[kalhauge]

@dhess, Thanks! I'm building it now.

edit: Same problem here.

virtual memory exhausted: Cannot allocate memory
`cc' failed in phase `C Compiler'. (Exit code: 1)
ghc/ghc.mk:111: ghc/stage1/package-data.mk: No such file or directory
make[1]: *** [utils/ghc-cabal/ghc.mk:48: utils/ghc-cabal/dist/build/tmp/ghc-cabal] Error 1
make: *** [Makefile:130: all] Error 2

Trying with 3 gb of swap!

[kalhauge]

Hi @dhess, It totally worked.

It did take around 10 days to compile but it actually worked. Big thanks from here!

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[Mic92]

@kalhauge seemed to solve the issue.