Nixpkgs: Clang can't find C++ header files

[abbradar]

Issue description

cc-wrapper's add-flags.sh sets NIX+CXXSTDLIB_COMPILE to an empty value. Because of this, fallback in cc-wrapper.sh doesn't work:

        NIX_x86_64_unknown_linux_gnu_CFLAGS_COMPILE+=" ${NIX_x86_64_unknown_linux_gnu_CXXSTDLIB_COMPILE:--isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0 -isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0/x86_64-unknown-linux-gnu}"

Steps to reproduce

 nixpkgs git:(master) ✗ nix-build -A clang
/nix/store/mwhqir1mj7l9n79amfq0gmwlw9pc6afh-clang-wrapper-4.0.1
 nixpkgs git:(master) ✗ result/bin/clang++ -std=c++11 test.cpp 
test.cpp:1:10: fatal error: 'iostream' file not found
#include <iostream>
         ^~~~~~~~~~
1 error generated.

Technical details

• Nixpkgs version: (run nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion) e458c44c2b7b2815104d4bf9894a08a41ae0cf6d

cc @Ericson2314 . I'm unsure how to fix it -- we can use ${foo:-default} substitution for the fallback instead but it will then incorrectly handle empty case (when someone really wants us to not add those flags).

[LnL7]

I thought default_cxx_stdlib_compile wasn't actually used anymore with https://github.com/NixOS/nixpkgs/pull/29246

[abbradar]

@LnL7 Ah, then that problem is Linux-specific -- on Linux we dont set NIX_CXXSTDLIB_* at all and expect fallback from cc-wrapper to be applied.

EDIT: I mean not because of default_cxx_stdib_compile specifically but because on Darwin libstdcxxHook is used which sets those variables, so this bug does not manifest.

[Ericson2314]

Let's use the hook everywhere then? Eventually I want to use pkg-config to handle per-package non-standard -L and -isystem flags, but until then I think using the hook everywhere is a good consistent stopgap.

[abbradar]

@Ericson2314 The hook only is a bad idea because then compilers won't work outside of nix-shell.

[Ericson2314]

@abbradar non-nix-shell/derivation usage is already pretty broken with the non-standard things we do. And the way this works on most distros---hard-coding a bunch of standard-library-specific information into the compiler itself is also unsavory.

The only good solution I see would a ccWithPackages like ghcWithPackages: basically do what the cc-wrapper and setup.sh accomplish within a wrapper script.

[abbradar]

I like ccWithPackages but feel that we should not completely break out-of-shell compilers either, and breaking standard library counts as this. It feels inelegant to me that our "wrapper" that traditionally is used to make a piece of software work would be "incomplete" in sense that software still can't be used. Also, I don't know of any other compiler or interpreter that we have that can't be used on NixOS if without extra libraries (e.g. to run vanilla scripts or small benchmarks -- of course (ccWithPackages clang []) could be used for that but still... feels good to have an intuitive meaningful default).

[abbradar]

@Ericson2314 After more thought, I'd say what seems a good solution to me is:

clang-unwrapped = oldPkgs.clang;
clang = ccWithPackages clang-unwrapped [];

, and stdenv uses clang-unwrapped. What do you think of it?

[Ericson2314]

@abbradar Err are you using -unwrapped with respect to the proposed ccWithPackages wrapper, or existing cc-wrapper? If you mean the former, yeah I think we are thinking the same thing.

[abbradar]

@Ericson2314 The former yeah -- oldPkgs is pkgs before the proposed change (with broken CXXSTDLIB).

[Ericson2314]

@abbradar cool, that sounds good to me then. My concern about the maintainability of stand-alone clang with with the actual raw binary. I agree that some derivation one can install and use without a curated environment is good thing.

[Ericson2314]

BTW, it might make sense to take a look at this at NixCon :).

[abbradar]

I plan to do just that 👍

[abbradar]

@Ericson2314 are you at the hackaton?

[arximboldi]

I am having a similar instance of this. Given this in my shell.nix file:

stdenv.mkDerivation rec {
  name = "abl-live-dev";
  buildInputs = [
    ccache
    git
    libcxx
    libcxxabi
    llvm_4
    ninja
    python
    clang_4
  ];

Then

NIX_CXXSTDLIB_COMPILE= -isystem /nix/store/jy2nwdl03d20wnjk7hrfksqazx46b2wg-libc++-4.0.1/include/c++/v1-isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0 -isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0/x86_64-unknown-linux-gnu

Notice how the second -isystem flag has somewhat ended up being merged into the previous one, with the sad result of the path being broken and Clang not being able to find the package. Is there a workaround this?

[LnL7]

@arximboldi You can use llvmPackages_4.stdenv.mkDerivation or clang_4.cc depending if you want to build the project with clang or just link against libclang.

[arximboldi]

Thanks for the tip @LnL7! But isn't the behaviour I am getting a bug nonetheless? it seems to be correct when I use an older <nixpkgs>, this is what I get with d0d905668c010b65795b57afdf7f0360aac6245b (good):

NIX_CXXSTDLIB_COMPILE= -isystem /nix/store/k4gavkplcyqyd29gp6fd9gain0p2cciz-libc++-4.0.1/include/c++/v1

This is what I get with <nixpkgs> from revision 0e4be9e5f07c8688abaf3f06b6976a662b993ac7 (broken)

NIX_CXXSTDLIB_COMPILE= -isystem /nix/store/jy2nwdl03d20wnjk7hrfksqazx46b2wg-libc++-4.0.1/include/c++/v1-isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0 -isystem /nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/include/c++/6.4.0/x86_64-unknown-linux-gnu

Or am I misunderstanding something else?

[arximboldi]

Mmmm ok, so actually with llvmPackages_4.stdenv.mkDerivation I get the same (broken) output in both cases. With the output that I marked as "good" it is not really that good since it actually fails to find some stuff inside "cstdlib.h", probably becaus the second line pointing to the gnu library is actually needed... :/

[arximboldi]

Sorry for the noise, using libcxxStdenv on d0d905668c010b65795b57afdf7f0360aac6245b seems to improve the situation...

[lambdafu]

With clang_4.cc it works for me, but I have no idea what I am doing. It's the third time my compiler broke in NixOS in the last half a year or so. :crying_cat_face:

[LnL7]

The problem with that is that it only works in a gcc based stdenv.

[gsauthof]

I can reproduce the issue with a newer clang and a small test program that includes iostream:

$ nix-env -i clang-wrapper-5.0.1
$ clang++ test.cc -g -Wall
test.cc:2:10: fatal error: 'iostream' file not found
#include <iostream>
         ^~~~~~~~~~
1 error generated.

With current GCC it works as expected:

$ nix-env -i gcc-wrapper-7.2.0
$ g++ test.cc -g -Wall
$ echo $?
0

[dtzWill]

1) Installing development packages is generally not functional with nixpkgs-- instead, enter a nix-shell with your requirements as inputs (or the convenient nix-shell -p clang_5 syntax). This should work on recent nixpkgs, I added tests of this usage a while back so hopefully this stays working.
(This includes using things like using a clang stdenv as others have discussed)

2) AFAICT the issues about borked NIX_CXXSTDLIB_COMPILE are now resolved. Please report if that's not the case so I can take a look! :)

3) Improving the situation for installed clang would be nice, for a number of reasons including easier use of libclang. Suggestions welcome, if I get some time I'll see what I can come up with here.
Maybe have it act differently when installed than its normal desired behavior of entirely relying on wrappers?

EDIT:
4) FWIW recently-ish libclang is in clang_5.cc.lib instead of clang_5.cc.

[gsauthof]

@dtzWill thanks for the hint, the development package situation wasn't obvious to me. And nix-shell -p clang_5 works for me.

Is it possible to tell nix-shell to configure clang++ with libc++ instead of libcstdc++?

[LnL7]

Yes, we have llvmPackages.libcxxClang and llvmPackages.libcxxStdenv.

[stephane-rolland]

Hi. I understand the problem should be fixed somewhere as @LnL7 said. However I still have the same issue. I'm using version 17.09.

I launch nix-shell with command:
$nix-shell -p llvmPackages.libcxxClang llvmPackages.libcxxStdenv clang_5
I have tried to compile a test main.cpp file
#include <iostream> int main(){ cout << "test";}
with the clang commands:
clang++ -Wall -pedantic -std=libc++ -c main.cpp
I get the following error

error: invalid value 'libc++' in '-std=libc++'

I have tried with -std=libcxx and get the same error, but with invalid value libcxx

On the contrary if I use -std=C++14 or -std=C++11, I get a different error:

fatal error: iostream file not found

What do you recommend ? That I change from version 17.09, or am I missing something ?

[LnL7]

-p always uses stdenv it's roughly equivalent to an expression like this.

stdenv.mkDerivation {
  name = "shell-environment";
  buildInputs = [ llvmPackages.libcxxClang llvmPackages.libcxxStdenv ];
}

Since that doesn't use libcxxStdenv.mkDerivation it has the same problems as mentioned above.

[langston-barrett]

When working with plain C (not C++), the following worked for me (in a shellHook):

    export CLANG="${clang_4}/bin/clang -I ${llvmPackages_4.clang.libc_dev}/include"

And for C++, the following worked:

with import <nixpkgs> { }; llvmPackages_6.libcxxStdenv.mkDerivation {
  name        = "test-iostream";
  src         = lib.sourceFilesBySuffices ./. [".cpp"];
  buildPhase  = ''
    clang -isystem ${llvmPackages_6.libcxx}/include/c++/v1 -O1 -g -emit-llvm -c *.cpp
  '';
  installPhase = ''
    mkdir -p $out
    mv *.bc $out
  '';
}

[siriobalmelli]

this may also be of help

[hlandau]

This issue still seems to be occurring.

nix-shell -p clang
$ echo '#include <cstddef>' > x.cpp
$ clang ./x.cpp
./x.cpp:1:10: fatal error: 'cstddef' file not found
#include <cstddef>
         ^~~~~~~~~
1 error generated.

Also happens inside a clangStdenv.

[matthewbauer]

@hlandau You need to use clang++

[tex]

I think this can be also closed as fixed by #54624.