Nixpkgs: Adding Clang to the buildInputs breaks G++/GCC linkage step

[forGGe]

Issue description

Clang, when specified as a dependency in buildInputs breaks G++ linkage, by introducing unrecognized flags for GNU linker via NIX_CXXSTDLIB_LINK environment variable.

Steps to reproduce

1. Create shell.nix:

   { pkgs ? import <nixpkgs> {} }:
   
   pkgs.stdenv.mkDerivation rec {
      name = "clang_with_gcc_test";
      buildInputs = [ pkgs.gcc6 pkgs.clang ];
   }
   

2. Launch shell:

   nix-shell ./shell.nix
   

3. Try to compile test C++ program:

   $ echo "int main() { }" > test.cpp && g++ test.cpp    
   

4. Observe output:

   g++: error: unrecognized command line option ‘-stdlib=libstdc++’; did you mean ‘-static-libstdc++’?
   

Technical details

Seems that it is a clang's setup hook that exposes that flag:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/llvm/4/libc++/setup-hook.sh#L3

So, issue can be workarounded either by removing Clang from buildInputs, or adding own hook to shell.nix:

shellHook = ''
    export NIX_CXXSTDLIB_LINK=""
'';

• System: ArchLinux:

  $ uname -a
  Linux Shakuras 4.12.12-1-ARCH #1 SMP PREEMPT Sun Sep 10 09:41:14 CEST 2017 x86_64 GNU/Linux       
  

• Nix version: 1.11.15

• Nixpkgs version: 18.03pre116475.d757d8142e
• Sandboxing enabled: no

P.S.

Thanks to @rasendubi for helping me figure out why it behaves like that!

[rasendubi]

Here is a one-file reproduce.

clang-gcc.nix:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation rec {
  name = "clang_with_gcc_test";
  buildInputs = [ pkgs.clang ];

  src = pkgs.writeTextDir "test.cpp" "int main() {}";

  buildPhase = ''
    g++ test.cpp -o $out
  '';

  dontInstall = true;
}                                                                                     

Steps to reproduce:

nix-build clang-gcc.nix

[orivej]

This behavior was introduced in #28998 and could also be worked around by using llvmPackages.clang-unwrapped as in #29333.

[orivej]

@LnL7, @Ericson2314
Shouldn't the clang-specific flags be added in a clang wrapper, rather than in its setup hook?

[forGGe]

Offtopic: excuse me for so many references to this issue, since I've falsely put issue number into the commit message.

[LnL7]

@orivej Is there a reason to have 2 wrapped compilers in a build environment? (and use both of them)

When including the clang wrapper clang and clang++ will work as expected, same for builds that use cc or c++. Otherwise the build probably only links against libclang or something in which case the unwrapped version should be used. (the wrapper doesn't have any libraries)

[Ericson2314]

Might be relevant, but @LnL7 made a gccStdenv, if one wants to use gcc on Darwin. Agreed that while this situation should ideally be solved, it's easy and correct to just avoid it in practice.

Long term, @LnL7 and I also talked about having a stdenv.cc.libcxx. Then wrapped clang, and wrapped clang alone, could always be pass a -stdlib to that, solving this problem completely.

[orivej]

@LnL7 Currently three cases are not supported:

1. Adding clang to build inputs yet using gcc.
2. Using clang and gcc compilers in the same derivation.
3. Installing clang into the system (and using it outside a derivation to compile C++ code).

So far there seems to be no need for the second case. The first is just inconvenient (and may be improved by exposing llvmPackages.clang-unwrapped as libclang). However, the third is valuable, and if we support it, this will likely add support for the other two.

(As an aside, include-what-you-use also requires manual specification of those NIX_CXXSTDLIB_COMPILE flags from libstdc++-hook.sh on the command line. This is not likely to be improved by changes to clang.)

@Ericson2314 Binding the wrapped clang to libc++ sounds perfect.

[ekmecic]

Per request of @orivej I'm mentioning how this issue is affecting my workflow.

I use irony and rtags for my C/C++ development. The basic principle of these tools is that they use libclang to process your source code for IDE-like functionality, i.e. autocomplete, jump-to-definition, find refs etc. To process your project, it must be compiled by clang.

The problem is that clang doesn't have a standard library in it's search path, so if your project has (as an example) #include <iostream> in it, it will fail to compile and the tools will fail to work as a result. I've developed a workaround for this, but it's janky and typically breaks on system updates.

The ideal situation for me is one where I can use both gcc and clang standalone from my regular command line. This was I can compile my projects with gcc and clang, but also use clang as the backend for my dev tools. Right now this is impossible without workarounds.

[robertodr]

I am in the same pickle as @ekmecic I actually need to be able to use both clang and gcc within the same nix-shell (@orivej second scenario) This is to compile a mixed Fortran/C++ project. While using clang++ and gfortran works, I also need to test that g++ and gfortran will work.

[tex]

I'm in the same pickle. Need to have g++ and clang++ to do C++ developement.
@ekmecic What is your workaround? Can you share it please?

[forGGe]

@ekmecic What is your workaround? Can you share it please?

See:
https://github.com/NixOS/nixpkgs/issues/29877#issuecomment-332723299

[tex]

That fixes it just partially (I'm on nixpkgs' master)

shell.nix:
{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation rec {
    name = "clang_with_gcc_test";
    buildInputs = [ 
      pkgs.gcc6
      pkgs.llvmPackages.clang-unwrapped
      pkgs.llvmPackages.llvm
    ];
}

test/main.cpp
#include <iostream>
int main(int argc, char **argv) {
  std::cout << 42;
  return 0;
}

nix-shell shell.nix
[nix-shell:~/dev/cpp]$ clang++ test/main.cpp 
test/main.cpp:1:10: fatal error: 'iostream' file not found
#include <iostream>
         ^~~~~~~~~~
1 error generated.

g++ works correctly and compiles the example.

[tex]

@ekmecic writes about this problem and that he has developed some workaround for it...

[siriobalmelli]

I've had this same problem, and I solved it in my toolbench.

1. gcc and clang coexisting peacefully: https://github.com/siriobalmelli/toolbench/blob/master/default.nix

2. YCM working using a custom .ycm_extra_config.py ... documented here

3. I use this toolchain e.g. to hack on my library nonlibc

[tex]

Interestingly this "works". g++ and clang++ are able to compile and link correctly.

Yes, gcc6 version is 32bit only and clang_6 version is 64bit version only. Don't ask me why, I do not understand why one is 32bit and other 64bit, I would expect both to be 32bit versions - pkgsi686Linux...

pkgs.pkgsi686Linux.stdenv.mkDerivation rec {
name = "clang_with_gcc_test";
buildInputs = [
gcc6
clang_6
];
}

[matthewbauer]

Can someone confirm this is still an issue?

[tex]

I believe it is still an issue. clang brings that flag that breaks gcc (g++: error: unrecognized command line option '-stdlib=libstdc++'; did you mean '-static-libstdc++'?) and clang-unwrapped cannot find system includes (fatal error: 'stdio.h' file not found).

[tex]

@matthewbauer I'd like to try it, but I do not know how to trigger rebuild of clang (as only .sh is changed). Is there any trick how to do it? I'm running NixOs on master.