Nixpkgs: [haskell] locally built ghc still suffers from "libz.so" problem while prebuilt ghc overcame it already?

[complyue]

this is a question other than bug report, I have to create an issue to ask this as I've searched hard but found no one to reveal how the problem had been resolved, and I need help to distribute an experimental ghc branch as a runtime dep for my haskell package. I found nixpkgs works perfectly to auto fetch&build&install my ghc branch's sdist tarball, except this problem, which still persists in latest nixpkgs manual, as:

15.9.4.7. Builds using Stack complain about missing system libraries

though I verified the prebuilt ghc 8.6.5 does NOT suffer this anymore, while the one I built locally from exactly the same source does.

any tip is appreciated, thanks in advance!

[complyue]

I have difficulty in understanding this statement:

Consequently, Cabal won’t record any search paths for libz.so in the package description, which means that the package works fine inside of nix-shell, but once you leave the shell the shared object can no longer be found.

does Cabal above mean cabal the library bundled as one of core libs in ghc?

I run both cabal v2-build and stack build inside a nix-shell, with their nix integration disabled, both succeeded with the pre-built ghc865, but failed at missing lib about "-lz" with a ghc865 I built locally with nix using an overlay like this:

https://github.com/complyue/hadui/blob/nix/nixpkgs-overlays/ghc865ife-overlay.nix

I've tested official sdist by replacing url&sha256 there, same result.

It can be reproduced with cabal get websockets FYI.

[complyue]

FYI this is how I'm going to distribute my Haskell tooling package with Nix: https://github.com/complyue/hadui/blob/nix/default.nix

I figured it out from scattered pieces of info many where, reviews and/or suggestions will be much appreciated!

[cdepillabout]

It sounds like you've fixed this problem? I'll go ahead and close this issue. If this isn't the case, please feel free to say something and I will reopen.

[complyue]

not solved, sorry I didn't make myself clear. current situation is prebuilt ghc works but locally built ghc does not, there is at least a reproducibility issue about building GHC with Nix.

[cdepillabout]

@complyue Sorry for closing too early. I've reopened now.

I'm having trouble understanding what the problem you're seeing is. Could you include a list of steps someone could follow to reproduce the problem you're experiencing?

[complyue]

sure, I'm to prepare a detailed reprod steps, will need a couple of hours as each GHC 8.6 build takes about an hour, and I woud simplify it to use packages already in Nix only.

[complyue]

@cdepillabout I can only access my mac desktop tomorrow where I had most investigations done, while I'm trying to reproduce the case with my linux laptop, it appears I might have used wrong cmdl to build things, now I come with a quick reproduction steps for my new findings:

Using the unstable channel of nixpkgs:

cyue@cyuembpx:~$ env | grep NIX
NIX_PROFILES=/nix/var/nix/profiles/default /home/cyue/.nix-profile
NIX_PATH=/home/cyue/.nix-defexpr/channels
NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
cyue@cyuembpx:~$ ls -lh ~/.nix-defexpr/channels/*
lrwxrwxrwx 1 cyue cyue 60 1月   1  1970 /home/cyue/.nix-defexpr/channels/manifest.nix -> /nix/store/8xyjvi9bx26l9qbvnrcjpzq8vsvq5c9x-env-manifest.nix
lrwxrwxrwx 1 cyue cyue 86 1月   1  1970 /home/cyue/.nix-defexpr/channels/nixpkgs -> /nix/store/dmdqvlxfk7c5gj4jg8iv77d8bmyk3c12-nixpkgs-20.03pre199995.895874d2145/nixpkgs
cyue@cyuembpx:~$ 

Taking latest websockets-0.12.6.1 from hackage as the build target:

cyue@cyuembpx:~$ mkdir /nix/issue73443
cyue@cyuembpx:~$ cd /nix/issue73443/
cyue@cyuembpx:/nix/issue73443$ cabal get websockets
Unpacking to websockets-0.12.6.1/
cyue@cyuembpx:/nix/issue73443$ cd websockets-0.12.6.1/

This method fails:

cyue@cyuembpx:/nix/issue73443/websockets-0.12.6.1$ nix-shell '<nixpkgs>' -A 'haskellPackages.websockets'

[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ cabal v2-build
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - websockets-0.12.6.1 (lib) (first run)
Configuring library for websockets-0.12.6.1..
Preprocessing library for websockets-0.12.6.1..
Building library for websockets-0.12.6.1..
[ 1 of 16] Compiling Network.WebSockets.Connection.Options ( 
 ...
[16 of 16] Compiling Network.WebSockets ( src/Network/WebSockets.hs, /nix/issue73443/websockets-0.12.6.1/dist-newstyle/build/x86_64-linux/ghc-8.6.5/websockets-0.12.6.1/build/Network/WebSockets.o )
/nix/store/cg0k49h66nkdqx6ccwnqr0i4q0fnfznc-binutils-2.31.1/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ exit
cyue@cyuembpx:/nix/issue73443/websockets-0.12.6.1$ 

This method succeeds:

cyue@cyuembpx:/nix/issue73443/websockets-0.12.6.1$ nix-shell -E 'with import <nixpkgs> {}; haskellPackages.shellFor {packages=p: [p.websockets];}'

[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ cabal v2-clean

[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ cabal v2-build
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - websockets-0.12.6.1 (lib) (first run)
Configuring library for websockets-0.12.6.1..
Preprocessing library for websockets-0.12.6.1..
Building library for websockets-0.12.6.1..
[ 1 of 16] Compiling Network.WebSockets.Connection.Options ( src/Network/WebSockets/Connection/Options.hs, /nix/issue73443/websockets-0.12.6.1/dist-newstyle/build/x86_64-linux/ghc-8.6.5/websockets-0.12.6.1/build/Network/WebSockets/Connection/Options.o )
 ...
[16 of 16] Compiling Network.WebSockets ( src/Network/WebSockets.hs, /nix/issue73443/websockets-0.12.6.1/dist-newstyle/build/x86_64-linux/ghc-8.6.5/websockets-0.12.6.1/build/Network/WebSockets.o )

[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ 

I used to assume nix-shell -A does the same as nix-shell -E with haskellPackages.shellFor, so clearly wrong since the above result shows.

it's very prolly I was failed by nix-shell -A while shellFor could have worked me out, in which case the original suspicion is plain wrong and this issue should be closed.
(btw I just saw your https://github.com/cdepillabout/post-about-nix-and-haskell/blob/master/2019-08-03-q-and-as-about-nix-for-haskellers.md and https://github.com/cdepillabout/nix-cabal-example-project, I regret I haven't discovered earlier, so delightful life-saving)

I'm still building local GHC to make progress in my linux env, will have you posted about any progress.

[complyue]

unfortunately seems I come back to the original state of this issue in my linux env, my local project fails build when my overlay for custom GHC is effective, while it builds okay without that overlay.

I reduced my user profile env, deleted old generations of it, then fired nix-collect-garbage. then I set substitute = false in nix.conf, and now launching nix-shell, it's building from source for many packages besides GHC now, I'd live it spinning for tonight and continue tomorrow.

[cdepillabout]

@complyue

You're conclusions sound correct.

The following:

$ nix-shell '<nixpkgs>' -A 'haskellPackages.websockets'

doesn't work because nix-shell only provides the direct dependencies to the environment it creates for you. It appears that websockets doesn't directly depend on zlib. One of the dependencies of websockets (I think streaming-commons?) depends on zlib.

However, either of the following should work:

nix-shell -E 'with import <nixpkgs> {}; haskellPackages.shellFor {packages=p: [p.websockets];}'

Or, the equivalent:

$ nix-shell -A haskellPackages.websockets.env '<nixpkgs>'

If you look at the source code, you can see that haskellPackages.websockets.env is using shellFor under the hood:

https://github.com/NixOS/nixpkgs/blob/4388c3190ebd0086da7b0b308aa3f689658339bd/pkgs/development/haskell-modules/generic-builder.nix#L480-L483

shellFor knows to pull out _all_ system dependencies for _all_ transitive dependencies and make them available in the environment.

Here are the lines that do this:

https://github.com/NixOS/nixpkgs/blob/4388c3190ebd0086da7b0b308aa3f689658339bd/pkgs/development/haskell-modules/make-package-set.nix#L284

https://github.com/NixOS/nixpkgs/blob/4388c3190ebd0086da7b0b308aa3f689658339bd/pkgs/development/haskell-modules/make-package-set.nix#L296

https://github.com/NixOS/nixpkgs/blob/4388c3190ebd0086da7b0b308aa3f689658339bd/pkgs/development/haskell-modules/make-package-set.nix#L310

---

my local project fails build when my overlay for custom GHC is effective, while it builds okay without that overlay

Could you add some more information about this? I'm having trouble understanding what is going on, or how to reproduce this issue you're seeing.

[complyue]

I verified that ghc865 fully built from source works without libz error, and as disabling my overlay can also get it pass, the culprit must reside in the way I crafted my overlay, and I further narrowed to two renamings, the 2 lines of comment starting with # ***:

# This is the Nix overlay to arm an experimental branch of GHC
# as the default compiler, together with the full Haskell package
# set (i.e. `haskellPackages`) with it.
#
# The compiler will be compiled from a source distribution, full
# source of the branch is at this repository:
#   https://gitlab.haskell.org/complyue/ghc/tree/ghc-8.6-ife
self: super:
let

  # mock the env at (<nixpkgs> + /pkgs/top-level/haskell-packages)
  callPackage = super.newScope {
    haskellLib = super.haskell.lib;
    overrides = super.pkgs.haskell.packageOverrides;
  };

  # the compiler
  compiler865ife = super.haskell.compiler.ghc865.overrideAttrs (oldAttrs: {

# use the official tarbal for reprod in
#   https://github.com/NixOS/nixpkgs/issues/73443
# src = super.fetchurl {
#   url = "https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-src.tar.xz";
#   sha256 = "0qg3zsmbk4rkwkc3jpas3zs74qaxmw4sp4v1mhsbj0a0dzls2jjd";
# };

    # *** either this rename will fail the build
    name = "${oldAttrs.name}-ife";

    src = super.fetchurl {
      # *** or this rename will fail the build
      name = "ghc-8.6.5-ife-src.tar.xz";
      url =
        "https://gitlab.haskell.org/complyue/ghc-ife-sdist/raw/master/ghc-8.6.5-src.tar.xz";
      sha256 = "0wf5v1ry3rlwhbsxlvka3qscdb4jz4jn7w3jckvwysh0fm1bavs5";
    };
  });


  # the package set
  haskellPackages865ife =
    callPackage (<nixpkgs> + /pkgs/development/haskell-modules) {
      buildHaskellPackages = super.buildPackages.haskell.packages.ghc865ife;
      ghc = compiler865ife;
      compilerConfig = callPackage (<nixpkgs>
        + /pkgs/development/haskell-modules/configuration-ghc-8.6.x.nix) { };
    };

in {

  # put the compiler & package set to standard location
  haskell = super.haskell // {
    compiler = super.haskell.compiler // { ghc865ife = compiler865ife; };
    packages = super.haskell.packages // { ghc865ife = haskellPackages865ife; };
  };

  # make this the default Haskell package set
  haskellPackages = haskellPackages865ife;
  # well the default can still be further overridden by other overlays

}

the buildings for renamed official sdist and unrenamed custom sdist is still in progress, I'll post the result once they are built so I can test with them.

if you are interested to do the experiments I'm doing, this should get you a minimal working example:

cyue@cyuembpx:~$ mkdir /nix/issue73443
cyue@cyuembpx:~$ cd /nix/issue73443/
cyue@cyuembpx:/nix/issue73443$ cabal get websockets
Unpacking to websockets-0.12.6.1/
cyue@cyuembpx:/nix/issue73443$ curl -OL https://github.com/complyue/hadui/raw/nix/nixpkgs-overlays/ghc865ife-overlay.nix

cyue@cyuembpx:/nix/issue73443$ nix-shell -E 'with import <nixpkgs> {overlays=[(import ./ghc865ife-overlay.nix)];}; haskellPackages.shellFor {packages=p: [p.websockets];}'

[nix-shell:/nix/issue73443]$ cd websockets-0.12.6.1/
[nix-shell:/nix/issue73443/websockets-0.12.6.1]$ cabal v2-build

[cdepillabout]

@complyue I'm still having trouble understanding what you're trying to do, and what exactly is causing it to fail. Could you try explaining it using simpler language?

From what I can tell from your comments, it looks like _either_ of the following is causing the build to fail when you try to build websockets with cabal:

1. overriding the GHC derivation to rename it to something like ghc-8.6.5-ife
2. naming the source tar-ball something like ghc-8.6.5-ife-src.tar.xz

Is this correct?

For 1., I don't know why this could be happening. At what point are you seeing an error? What is the exact error?

A quick grep through the nixpkgs codebase makes it seem like ghc.name is being used a couple places to determine directory structure:

$ git grep 'ghc.name'
pkgs/development/haskell-modules/generic-builder.nix:    (optionalString enableSeparateDataOutput "--datadir=$data/share/${ghc.name}")
pkgs/development/haskell-modules/generic-builder.nix:    (optionalString (enableSharedExecutables && stdenv.isLinux) "--ghc-option=-optl=-Wl,-rpath=$out/lib/${ghc.name}/${pname}-${version}")
pkgs/development/haskell-modules/generic-builder.nix:      ${buildPkgDb ghc.name "$packageConfDir"}
pkgs/development/haskell-modules/generic-builder.nix:      local packageConfDir="$out/lib/${ghc.name}/package.conf.d"
pkgs/development/haskell-modules/with-packages-wrapper.nix:  name = ghc.name + "-with-packages";
pkgs/development/haskell-modules/with-packages-wrapper.nix:    local packageConfDir="$out/lib/${ghc.name}/package.conf.d";

I guess it is possible this could be causing an error?

Maybe you could try the build without renaming the compiler?

For 2., most of the time in nixpkgs, the srcs aren't given names. I'd suggest not adding a name here. Atlhough I'd be surprised if this ended up causing an error.

[cdepillabout]

Oh, also, one more thing:

It sounds like you're getting an error when building with cabal, but what if you just build with nix?

Instead of

$ nix-shell -E 'with import <nixpkgs> {overlays=[(import ./ghc865ife-overlay.nix)];}; haskellPackages.shellFor {packages=p: [p.websockets];}'

what about

$ nix-build -E 'with import <nixpkgs> {overlays=[(import ./ghc865ife-overlay.nix)];}; haskellPackages.websockets'

Does that work?

---

Also, just a small comment on your approach, but it might be easier to override one of the existing package sets than to create your own (I haven't tested this exact code, but I think something like this should work):

  haskellPackages865ife = super.haskell.packages.ghc865.override {
    # this might not need to be overridden
    buildHaskellPackages = super.buildPackages.haskell.packages.ghc865ife;
    ghc = compiler865ife;
  };

That way you also wouldn't have to redefine callPackage.

[complyue]

@cdepillabout all your understandings are right, I didn't expect renaming could be a problem at first , I still doubt it but the test result suggest that so far, my building not finished yet, will further confirm shortly later.

and sorry I couldn't express myself clearer, I'll try be simpler.

nix-build and nix-env -iA already worked earlier, I was just stuck at cabal build and stack build.

I did intend to override pkgs.haskell.packages.ghc865 instead of using callPackage, I'll try your suggestion and see result. but I roughly remember I did similar, and the built haskell package within the package set uses ghc865 instead of ghc865ife at runtime.

my final goal is to allow a user to install hadui with a simple command like nix-env -iA hadui -f https://github.com/complyue/hadui/archive/0.1.0.0.tar.gz. Hadui will call ghc at runtime, and it must find one built from the custom sdist. Update: to be clearer - Hadui will use ghc-ife at runtime to dynamically compile & execute Haskell code from a underlying project (nix/cabal/stack).

Hadui works roughly like this: it provides an executable hadui-dev, this executable does not much on itself, will finally exec stack ghci ... --ghci-options '-e ":frontend HaduiDev"' after reading its cfg. stack ghci ... in turn finally exec ghc --interactive ... and pass control to HaduiDev the specified ghc frontend plugin defined by Hadui at https://github.com/complyue/hadui/blob/nix/hadui/src/HaduiDev.hs

GHCi's :frontend command is only supported by my experimental ghc branch, the frontend plugin mechanism of GHC is very mature, but only works in non-interactive modes. I'm proposing the interactive support to GHC with this ticket: https://gitlab.haskell.org/ghc/ghc/issues/17348

[complyue]

@cdepillabout I'm still waiting my builds to finish, meanwhile I'm looking into impl of shellFor, here:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/make-package-set.nix#L302~L303

        ghcCommand' = if ghc.isGhcjs or false then "ghcjs" else "ghc";
        ghcCommand = "${ghc.targetPrefix}${ghcCommand'}";

as ghc here is from the closure capturing the outer most argument, super.haskell.packages.ghc865.override supposedly can't override it imho, I remember I turned to callPackage by myself bcoz of this. anyway it will be verified once my builds finished.

[cdepillabout]

@complyue Thanks for the additional explanation. I'm actually working at a machine-learning consultancy in Tokyo, so I applaud your effort to get more data-science-related tools for Haskell!

Let me make sure I understand what you're saying.

nix-build and nix-env -iA already worked earlier, I was just stuck at cabal build and stack build.

Ah, okay, so you have a derivation that is working when you build it with nix-build, but it is failing to build when you do something like nix-shell -p cabal-install -p ghc865ife and try to build hadui with that cabal and ghc?

my final goal is to allow a user to install hadui with a simple command like nix-env -iA hadui -f https://github.com/complyue/hadui/archive/0.1.0.0.tar.gz

I see. So you want to provide a default.nix file in the root of your hadui repo, and have users be able to easily use hadui-dev by building that?

Is there anything I can help you with in debugging why this isn't work? Is this the branch you are working from? https://github.com/complyue/hadui/tree/nix

---

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/make-package-set.nix#L302~L303

This is just an aside, but if you link directly to a commit (instead of a branch), GitHub will expand it into the code you're linking to.

For example,

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/make-package-set.nix#L302~L303

just becomes

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/make-package-set.nix#L302~L303

but

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/development/haskell-modules/make-package-set.nix#L302-L303

becomes

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/development/haskell-modules/make-package-set.nix#L302-L303

---

As for your question, I believe ghc comes from

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/development/haskell-modules/make-package-set.nix#L22-L23

which comes from

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/development/haskell-modules/default.nix#L17-L20

which comes from

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/development/haskell-modules/default.nix#L1-L10

which comes from

https://github.com/NixOS/nixpkgs/blob/66a9f39817c2054ad18d78063ae9a9d50d1f1ee3/pkgs/top-level/haskell-packages.nix#L113-L117

and should be able to be overridden.

If you're not cross-compiling, I don't think you should have to worry about the difference between buildPackages.haskell and the normal top-level haskell.

[complyue]

I verified that renaming the compiler is the culprit and renaming the tarball does no harm per my test case.

much thanks to @cdepillabout, being so gracious and helpful, actually not only you helped analyze the situation here I'm experiencing, I also benefit a lot after reading the https://github.com/cdepillabout/nix-cabal-example-project you shared, I establishes better practice and gained deeper understanding for internal details about haskelling with nix.

and for the concern led me to do my own callPackage, it's non-issue as either nix-shell -A 'haskellPackages.hadui.env' '/path/to/hadui' or nix-shell -E 'with import '/path/to/hadui' {}; haskellPackages.shellFor {packages=p: [p.hadui];}' will work out, as @cdepillabout suggested. it should be me to have omitted .env in the -A form earlier, to have caused the failure. by following all suggestions from @cdepillabout , all work so far so good.

thanks again to @cdepillabout , please close this issue whenever you see fit.

[complyue]

@cdepillabout I guess you and me typing the last comment nearly concurrently :) and I learned from you yet another new thing about github's inline code feature :D

and yes I'm currently working on hadui/nix branch to bring support for projects in all nix/cabal/stack formats, and use nix to get rid of the pain a user has to suffer from building ghc from source by him/her self, (though the build takes an hour, a sure-fire with expectable result is so much better than to be failed every half hour by dependency issues in building ghc), and nix is really amazing in this regard, and learning nix is also a pleasure for me.

speaking of Hadui, it's rather new and I'm open to all kind of collaborations around it, actually I'd like a more public oriented home for it (than under my personal name), before considering to publish it to hackage, you have suggestions about this?

and I'll be really glad if you are interested to be a maintainer together with me, it won't bear a growing feature set so I expect the burden of maintenance should be mild.

I live in mainland China, and recently VPNs are banned hard, google/facebook/twitter etc work occasionally even with VPN, fortunately gitlab/github/gitter are banned softer recent days so I can use these more time in a day than not at all.

[cdepillabout]

@complyue

I verified that renaming the compiler is the culprit and renaming the tarball does no harm per my test case.

Okay great, I also didn't know that renaming the compiler could cause problems like this, but it is good to know.

speaking of Hadui, it's rather new and I'm open to all kind of collaborations around it, actually I'd like a more public oriented home for it (than under my personal name), before considering to publish it to hackage, you have suggestions about this?

On GitHub, some people create an organization for a package and surrounding packages. For example, you could create an organization called hadui and have a single repository called hadui. I think a lot of users are used to this setup.

However, it sounds like the GFW is quite annoying to work around, so hopefully you find a good solution.

and I'll be really glad if you are interested to be a maintainer together with me,

Unfortunately I'm pretty busy with work and nix stuff, but I wish you luck with developing hadui! If you ever have any more nix questions, please feel free to open an issue here, or post on the nix discourse: https://discourse.nixos.org/