Nixpkgs: How to add further targets to rustc?

[nomeata]

I am playing around with building stuff for wasm32-unknown-unknown. Until now I was using the https://github.com/mozilla/nixpkgs-mozilla repository, but I noticed that my experiments also work with rustc-1.38, so I don’t actually need nightly. I wanted to change to using nixpkgs proper, but now I get

error[E0463]: can't find crate for `core`
  |
  = note: the `wasm32-unknown-unknown` target may not be installed

error: aborting due to previous error

With rustup I can do

rustup target add wasm32-unknown-unknown

but what do I have to do to get the same effect when I use pkgs.rustPlatform.buildRustPackage?

[samlich]

My impression is that the correct way is using something like pkgs.pkgsCross.wasi32.rustc, however you must use pkgs.pkgsCross.wasi32.buildPackages.rustc because wasm isn't supported in nixpkgs for rustc right now (at least I think that's why). However, that doesn't work because it's wasm32-unknown-wasi rather than wasm32-unknown-unknown (I think, it just fails on an unwrap()), and there's other fixes needed anyway. I have an overlay to add targets that I made before pkgsCross and just updated as I needed wasm again. I'm not sure how to convert it to the proper cross-compilation methods as wasm seems to want rust-lld which is included when using the included llvm, rather than the system one which is not ideal.

I also use this for building nightly, so I removed some parts. Not sure what openssl was needed for, but might as well leave it. The postUnpack may just be fixing nightly things and might even need to be removed. Emscripten can be removed, unless you want it. You may be able to build with the profiler, but I had some sort of problem at least on nightly. A lot of the preFixup is obviously unnecessary, but I smuggled out an unadulterated rust-lld when figuring out how to use it. You'll also need around 14 GiB of ram free and 12 GiB or so allocated to /tmp.

  rustcStable = super.rust.packages.stable.rustc;
  rustc = (rustcStable.overrideAttrs (oldAttrs: rec {
    buildInputs = oldAttrs.buildInputs ++ [ self.pkg-config self.openssl ];
    postUnpack = ''
    ls -R $sourceRoot/src/llvm-project/compiler-rt/
    rm -r $sourceRoot/src/llvm-project/compiler-rt/
    '';
    configureFlags = let add_flags = [
      "--enable-full-tools" # something about llvm, not related to --tools
      "--enable-emscripten"
      "--release-channel=nightly"
      "--target=x86_64-unknown-linux-gnu,wasm32-unknown-unknown,asmjs-unknown-emscripten"
      "--tools=" # don't build cargo, clippy, etc. as they break stuff
    ];
    in (builtins.filter (x: !builtins.any (y: x == y) ([
      "--enable-profiler" ["--enable-profiler"]
      "--release-channel=stable"
      "--target=x86_64-unknown-linux-gnu"
                         ] ++ add_flags)) ({ configureFlags = []; } // oldAttrs).configureFlags)
    ++ add_flags;

    preFixup = ''
    # mv $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld $out/bin/
    mkdir -p $out/etc/ || true
    tar -cf $out/etc/rust-lld.tar.xz $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    pwd
    tar -cf ./rust-lld.tar.xz $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true

    ldd $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    patchelf --print-interpreter $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true


    patchelf --print-rpath $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    patchelf --shrink-rpath $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    patchelf --print-rpath $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    patchelf --set-rpath \$ORIGIN/../lib:$out/lib:${self.stdenv.cc.libc}/lib:${self.stdenv.cc.cc.lib}/lib $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    patchelf --print-rpath $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld || true
    ln -s ../lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld $out/bin/rust-lld || true
    ln -s $out/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld $out/bin/rust-lld2 || true
    '';
  })).override {
    withBundledLLVM = true; # using wasm target uses rust-lld
  };

I would be interested in upstreaming these changes, if they've been done appropriately. Someone would need to give me a hint on rust-lld, unless it's fine to just compile the bundled llvm when wanting wasm support. There'd also need to be a wasm32-unknown-unknown system?

[nmattia]

I also fought with this a bit recently, here's a very hacky way of enabling an x86_64-unknown-linux-musl target: https://github.com/NixOS/nixpkgs/compare/NixOS:0002a7d...nmattia:5d31288

I'll try to clean it up and shoehorn it in the current cross compilation framework. I just could not get it to work by not using the bundled LLVM, and @TheSandwichMakr I see you have something similar here.

[nomeata]

I wasn’t expecting this to involve the full cross compiliation machinery; as I don't want to build arbitrary nix derivation for wasm32-unknown-unknown, but maybe it is the right thing to do (not my area of expertise). I’ll stick to the mozilla repository for the time being, and watch this space :-)

[nomeata]

I am experimenting with an overlay like

        (self: super: {
          rustc = super.rustc.overrideAttrs (old: {
        configureFlags = self.lib.lists.forEach old.configureFlags (flag:
              if self.lib.strings.hasPrefix "--target=" flag
              then flag + ",wasm32-unknown-unknown"
              else flag
            );
          });
        })

but direct support in nixpkgs would indeed be neat.

[nmattia]

FYI I just added a musl/wasm32 rustc to naersk: https://github.com/nmattia/naersk/pull/97
One of the next steps is to make it configurable.

EDIT: and same here, more flexibility in nixpkgs directly would be great.

[nomeata]

EDIT: and same here, more flexibility in nixpkgs directly would be great.

I actually don't want flexibility, I just want rustc in nixpkgs to support wasm32 target by default, so that I (or my contributers) don’t have to build rust and can get a working one from the official cache :-)

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[aaronjanse]

Related: https://github.com/NixOS/nixpkgs/pull/95542