Nixpkgs: Build error for llvm_5 on NixOS (unstable)

[Pauan]

Issue description

I am using NixOS (unstable) and I tried adding llvm_5 to my environment.systemPackages, but there were two issues:

1. It downloaded and compiled llvm_5 from source. Shouldn't it be downloading a pre-compiled binary instead?

2. I got this build error:

   ```
   [100%] Running all regression tests
   /nix/store/hr461qqci171p6k53wwz52ifyp2h9g9l-python-2.7.14/bin/python2.7 /tmp/nix-build-llvm-5.0.0.drv-0/llvm/utils/lit/lit.py -sv --param llvm_site_config=/tmp/nix-build-llvm-5.0.0.drv-0/llvm/build/test/lit.site.cfg --param llvm_unit_site_config=/tmp/nix-build-llvm-5.0.0.drv-0/llvm/build/test/Unit/lit.site.cfg /tmp/nix-build-llvm-5.0.0.drv-0/llvm/build/test
   -- Testing: 21404 tests, 8 threads --
   Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.
   FAIL: LLVM :: tools/llvm-ar/empty-uid-gid.test (20968 of 21404)
   ***** TEST 'LLVM :: tools/llvm-ar/empty-uid-gid.test' FAILED *****

   Script:

   /tmp/nix-build-llvm-5.0.0.drv-0/llvm/build/./bin/llvm-ar tv /tmp/nix-build-llvm-5.0.0.drv-0/llvm/test/tools/llvm-ar/Inputs/msvc-import.lib | /tmp/nix-build-llvm-5.0.0.drv-0/llvm/build/./bin/FileCheck /tmp/nix-build-llvm-5.0.0.drv-0/llvm/test/tools/llvm-ar/empty-uid-gid.test

   Exit Code: 1

   Command Output (stderr):

   /tmp/nix-build-llvm-5.0.0.drv-0/llvm/test/tools/llvm-ar/empty-uid-gid.test:3:8: error: expected string not found in input
   CHECK: --------- 0/0 0 1970-01-01 {{[0-9:.]+}} library.dll
   ^
   :1:1: note: scanning from here
   --------- 0/0 0 1969-12-31 22:00:00.000000000 library.dll
   ^

   --

---

Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 227.29s

---

Failing Tests (1):
LLVM :: tools/llvm-ar/empty-uid-gid.test

 Expected Passes    : 20587
 Expected Failures  : 131
 Unsupported Tests  : 685
 Unexpected Failures: 1

make[3]: * [CMakeFiles/check-all.dir/build.make:58: CMakeFiles/check-all] Error 1
make[3]: Leaving directory '/tmp/nix-build-llvm-5.0.0.drv-0/llvm/build'
make[2]: [CMakeFiles/Makefile2:158: CMakeFiles/check-all.dir/all] Error 2
make[2]: Leaving directory '/tmp/nix-build-llvm-5.0.0.drv-0/llvm/build'
make[1]: [CMakeFiles/Makefile2:165: CMakeFiles/check-all.dir/rule] Error 2
make[1]: Leaving directory '/tmp/nix-build-llvm-5.0.0.drv-0/llvm/build'
make: * [Makefile:199: check-all] Error 2
builder for ‘/nix/store/56agivwlq78dchv2kkyqpjzv7rqzfvyg-llvm-5.0.0.drv’ failed with exit code 2
cannot build derivation ‘/nix/store/k2na18wg9g5fzfmshwkzy8d3mcfywafh-system-path.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/0dl1y62h86v2md0cjcv8mmj2rd7y3f6g-nixos-system-nixos-18.03pre123588.ade98dc442e.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/0dl1y62h86v2md0cjcv8mmj2rd7y3f6g-nixos-system-nixos-18.03pre123588.ade98dc442e.drv’ failed
```

It looks like a really simple and harmless error, but it's preventing me from using llvm_5

P.S. My timezone is GMT-1000, which might be related to this.

Steps to reproduce

1. Add llvm_5 to environment.systemPackages in configuration.nix

2. Run nixos-rebuild switch

Technical details

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.9.70, NixOS, 18.03pre123130.1bc288591ea (Impala)`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 1.11.16`
 - channels(root): `"nixos-18.03pre123588.ade98dc442e"`
 - channels(pauan): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`

[dtzWill]

That's strange. Yes, I'd expect LLVM_5 to be in the cache!

Using each of the two git revisions shown in your nix-info results in the following paths forllvm_5:

• ade98dc442e:

$ nix-build https://github.com/NixOS/nixpkgs/archive/ade98dc442e.tar.gz -A llvm_5
/nix/store/4ipg4jy2fiw1fiqrdxdm6vv59jlr1qj4-llvm-5.0.0

• 1bc288591ea :

$ nix-build https://github.com/NixOS/nixpkgs/archive/1bc288591ea.tar.gz -A llvm_5
/nix/store/wdl31mxv2hqb5zkbfym7faqb2998d694-llvm-5.0.0

Both of which are in the binary cache (checked with some nix copy commands).

What's curious is neither of these have a deriver that matches the one for your failed build?

$ nix-store -q --deriver /nix/store/4ipg4jy2fiw1fiqrdxdm6vv59jlr1qj4-llvm-5.0.0
/nix/store/w53da9r8b0zk3c158w16x9xczml7b2f4-llvm-5.0.0.drv
$ nix-store -q --deriver /nix/store/wdl31mxv2hqb5zkbfym7faqb2998d694-llvm-5.0.0
/nix/store/s67d73ib98y43n2yahx0khd85zklq0jf-llvm-5.0.0.drv

And indeed your derivation (/nix/store/s67d73ib98y43n2yahx0khd85zklq0jf-llvm-5.0.0.drv) is not available (no build log).

Anyway what I'm getting at is:

• AFAICT something in your environment is causing llvm_5 to evaluate to something that doesn't match llvm_5 from either channel you're using... which is probably the same reason you're not getting a cached version. Your instincts here are good! :)
• I notice sandbox is disabled. Enabling it won't change the problem mentioned above /but/ it quite possibly will fix your build problem. Might be worth trying again with sandbox turned on, for lack of a better idea.

[dtzWill]

Okay so I tried adding llvm_5 to my environment.systemPackages as well and....

now I'm building /nix/store/q2yi3rmw32120n4mdsvjrgvn35jgppp3-llvm-5.0.0.drv :(

.... What? We'll see if tests pass...

[dtzWill]

Looks like it's the .man output, blargh.

FWIW, llvm_5.man derivations for the two commits:

• ade98...: /nix/store/2zn0qq47f3m1g87yalwc0vfqbq8cvnhg-llvm-5.0.0.drv
• 1bc...: /nix/store/26sdadc99f55wgpcljh38q4ava803m8d-llvm-5.0.0.drv

Honestly these LLVM/clang man pages might be more trouble than they're worth...

I suppose the question is now "why isn't this cached?".

EDIT: Answer appears to be "we only build and cache all of llvmPackages, so which is currently LLVM 4 (cc #33374)".
(we use recurseIntoAttrs for llvmPackages but not others; I'll submit a PR to have this changed...)

[LnL7]

We don't want to enable manpages by default because that brings in a bunch of extra dependencies. Perhaps it's better to make a separate drv for that to avoid confusion?

[vcunat]

It would be best if that derivation only built manpages (cheaply). Then we would add it to Hydra jobs, and you'd get both from cache.

[LnL7]

Yeah, that's what I was thinking of.

[dtzWill]

Proof-of-concept for building 'only' manpages: https://github.com/dtzWill/nixpkgs/tree/feature/llvm-manpages-slim
(llvm_4 only, llvm_5 should just be more of the same)

Bummer about #24717 -- perhaps we should just leave these as separate derivations (that need to be installed separately) instead of the current sometimes-you-get-manpages-but-only-if-you-install-to-system-or-something behavior?

[dtzWill]

Can this be closed now? I think the worst of this was mitigated by #33948 but let us know if that's not the case.