Nixpkgs: lua fails to build when cross-compiling

[kisik21]

Describe the bug
Due to weird hardcoded CC, AR and RANLIB in Makefiles, lua doesn't like to be cross-compiled. The build crashes, unable to find gcc.

To Reproduce
Steps to reproduce the behavior:

1. nix-build --arg crossSystem '{ system = "armv7l-linux" }' '<nixpkgs>' -A lua
2. Wait for the build to crash

Expected behavior
lua cross-compiles successfuly, like any other package

Screenshots
Here's an excerpt from the build log:

building '/nix/store/isd9w196grdcwgrs00fi3fv54bx31llp-lua-5.2.4-armv7l-unknown-linux-gnueabihf.drv'...
unpacking sources
unpacking source archive /nix/store/28g8gzsd38c3kli6s0f0p0q9a95s7vgm-lua-5.2.tar.gz
source root is lua-5.2.4
setting SOURCE_DATE_EPOCH to timestamp 1424910717 of file lua-5.2.4/README
patching sources
applying patch /nix/store/jpkm2nm2m9xf9m9v44cr5lw9kx3b2rf7-lua-arch.patch
patching file Makefile
patching file src/luaconf.h
patching file src/Makefile
source root is libzip-1.3.0
updateAutotoolsGnuConfigScriptsPhase
configuring
setting SOURCE_DATE_EPOCH to timestamp 1504368659 of file libzip-1.3.0/man/zip_stat.man
building
patching sources
build flags: SHELL=/nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin/bash INSTALL_TOP=/nix/store/j4faq0xc9a875n54w7aczgfhkadkklls-lua-5.2.4-armv7l-unknown-linux-gnueabihf INSTALL_MAN=/nix/store/j4faq0xc9a875n54w7aczgfhkadkklls-lua-5.2.4-armv7l-unknown-linux-gnueabihf/share/man/man1 R=5.2.4 LDFLAGS=-fPIC V=5.2 PLAT=linux CFLAGS=-DLUA_USE_LINUX\ -O2\ -fPIC\ -DLUA_COMPAT_ALL
cd src && make linux V=5.2 R=5.2.4
make[1]: Entering directory '/build/lua-5.2.4/src'
make all SYSCFLAGS="-DLUA_USE_LINUX" SYSLIBS="-Wl,-E -ldl -lreadline"
patching script interpreter paths in test-driver
make[2]: Entering directory '/build/lua-5.2.4/src'
gcc -DLUA_USE_LINUX -O2 -fPIC -DLUA_COMPAT_ALL   -c -o lapi.o lapi.c
/nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin/bash: gcc: command not found
make[2]: *** [<builtin>: lapi.o] Error 127
make[2]: Leaving directory '/build/lua-5.2.4/src'
make[1]: *** [Makefile:113: linux] Error 2
make[1]: Leaving directory '/build/lua-5.2.4/src'
make: *** [Makefile:55: linux] Error 2
test-driver: interpreter directive changed from " /bin/sh" to "/nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin/sh"
builder for '/nix/store/isd9w196grdcwgrs00fi3fv54bx31llp-lua-5.2.4-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 2

Additional context
I'm currently trying to build a usable server system for armv7l to put my old Raspberry Pi 2 to good use and reporting bugs with cross-compilation as I go. This is the first one I couldn't fix myself with a simple packageOverride.

Metadata

• system: "x86_64-linux"
• host os: Linux 4.19.62, NixOS, 19.09pre186857.239fffc90d7 (Loris)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.2.2
• channels(root): "cleveca22, nixos-19.09pre186857.239fffc90d7"
• channels(kisik21): "home-manager"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Maintainer information:

attribute:
   - lua
   - lua5
   - lua51
   - lua52
   - lua53

[vcunat]

You might want to try luajit as a quick work-around... only for some reason it needs build and host to have the same pointer size – but one can usually choose x86_64 or i686 as build (i.e. on -linux), e.g.

nix build -f . --arg crossSystem '(import ./lib).systems.examples.armv7l-hf-multiplatform' --argstr system i686-linux luajit

(tested the build now on current unstable: 8746c77a)

[kisik21]

@vcunat any ways to use that with packages such as nmap and prosody in the configuration.nix? (without rebuilding my whole system closure to cross-compile from i686 if possible)

[vcunat]

You can certainly import whole nixpkgs into a let-binding with differently passed parameters (system and crossSystem in this case) and use attributes from there, though I'm not sure what's the best-practice way of doing that... but import <nixpkgs> { parameters } should work fine IIRC.

[kisik21]

@vcunat with some hacking (it was very weird but I think I solved it now!) I seem to be able to carry on. As a workaround, I used the following overlay:

let
  target = "armv7l-linux";
  host = "i686-linux";
  fixedLuaJIT = (import <nixpkgs> {
    crossSystem = { system = target; };
    localSystem = { system = host; };
  });
in self: super: {
  # Replacing LuaJIT with a fixed version.
  luajit = fixedLuaJIT;
  # Some programs depend on Lua. We replace Lua here.
  # This list is obviously incomplete!
  highlight = super.highlight.override { lua = fixedLuaJIT; };
  prosody = super.prosody.override { lua5 = fixedLuaJIT; };
  # UPD: some sort of API incompatibility?
  #nmap = super.nmap.override { lua5_3 = fixedLuaJIT; };
}

[vcunat]

nmap: luajit fully supports 5.1 version of the language, though it does now have some features from later versions – I suppose that's why it won't (directly) work there.

[kisik21]

I fixed it with an overlay!

let
  fixedLua = lua: stdenv: (lua.overrideAttrs (oldAttrs: {
    makeFlags = oldAttrs.makeFlags ++ [
      "CC=${stdenv.hostPlatform.config}-gcc"
      "RANLIB=${stdenv.hostPlatform.config}-ranlib"
    ];
  })).overrideAttrs (oldAttrs: {
    configurePhase = ''
      makeFlagsArray+=("AR=${stdenv.hostPlatform.config}-ar rcu")
    '';
  });
in self: super: {
  lua5_1 = fixedLua super.lua5_1 self.stdenv;
  lua5_2 = fixedLua super.lua5_2 self.stdenv;
  lua5_2_compat = fixedLua super.lua5_2_compat self.stdenv;
  lua5_3 = fixedLua super.lua5_3 self.stdenv;
  lua5_3_compat = fixedLua super.lua5_3_compat self.stdenv;
}

I'll make a patch to nixpkgs very soon and close the issue myself! (could someone assign this to me if possible?)

[teto]

thanks for working on this.

[kisik21]

@teto I'm doing this because of my reasons, but why not contribute? armv7l-linux cross-compile needs more love and I'm gonna provide it! Expect more patches and issues coming ahead as I'm reporting my findings while trying to build a working system :3

[kisik21]

Sent a PR - #66883. Will close when it gets merged. Or should I close it now?