Nixpkgs: neovim fails to build on master (due to lua / luarocks)

Created on 11 Feb 2020  Â·  15Comments  Â·  Source: NixOS/nixpkgs

Describe the bug
neovim no longer builds on master (tested f77e057cda60a3f96a4010a698ff3be311bf18c6), broken since commit d4f09b65906c1077397da825671f8b35e0175fdd (by git bisect, ping @doronbehar, author of latest luarocks update).

$ nix-build -A neovim
these derivations will be built:
  /nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv
  /nix/store/3adbwmw56z7qgjqzm7vw7kqxr5hmflw6-neovim-unwrapped-0.4.3.drv
  /nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv
building '/nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.30.0-0 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/3fncfqvh6mz5x2my2s3aiqa9w0rmvcy4-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/f711zxf89ga3in4gph12jclqdryc5n34-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/vy5x75rdxcr8g4jk2xv31zvd9mfcyb6v-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1.30.0
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory

Error: Failed copying /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so to /nix/store/d29kmg3nijdi4s3wb0k68kvfz3nq3frf-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so
builder for '/nix/store/4wx9xn4zysrqkia9n55vb46jfxpi6a99-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed with exit code 1
cannot build derivation '/nix/store/3adbwmw56z7qgjqzm7vw7kqxr5hmflw6-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/3yi4yxsws4g3qyjjphhsbx1hlicwnf77-neovim-0.4.3.drv' failed

Expected behavior
I expect neovim to (continue to) build when I upgrade my system.

bug regression lua vim
Source
picture bjornfor
👍10

Most helpful comment

I can confirm that after overriding luv with 1.34.1-1 as below I can build neovim again.

{
      version = "1.34.1-1";
      src = self.fetchurl {
        url = https://luarocks.org/luv-1.34.1-1.src.rock;
        sha256 = "044cyp25xn35nj5qp1hx04lfkzrpa6adhqjshq2g7wvbga77p1q0";
      };
    }
picture ArdaXi on 17 Feb 2020
🎉3

All 15 comments

Warning: unmatched variable LUA_LIBDIR

This seems to be the root cause of the issue.

That's a hard issue to debug. What I've done in the meantime is test luarocks install luv (with the right environment from a shell.nix) and it ran flawlessly. Moreover, I couldn't figure out the difference between libluv and luv - they both install different .so files but I can't point a difference between the two upstream.

I've opened an issue on Luarocks' end: https://github.com/luarocks/luarocks/issues/1155 .

Besides that, I noticed that https://github.com/luvit/luv (whether that's libluv or luv) is outdated - we have 1.30.0 and they have 1.34.1. @teto perhaps we should update it so we'd be more confident the issue wasn't fixed upstream and hasn't landed here yet?

I'm ccing @manveru, @rvolosatovs and @Ma27 as the maintainers of Neovim which is the sole user of lua.pkgs.libluv.

doronbehar picture doronbehar on 12 Feb 2020

updating luv is definitely worth a try as their build system required us to do some contortions. Hopefully they've since then made some changes that improve things.

teto picture teto on 12 Feb 2020

I can confirm that after overriding luv with 1.34.1-1 as below I can build neovim again.

{
      version = "1.34.1-1";
      src = self.fetchurl {
        url = https://luarocks.org/luv-1.34.1-1.src.rock;
        sha256 = "044cyp25xn35nj5qp1hx04lfkzrpa6adhqjshq2g7wvbga77p1q0";
      };
    }
ArdaXi picture ArdaXi on 17 Feb 2020
🎉3

@bjornfor please try on latest master after #80369

mjlbach picture mjlbach on 17 Feb 2020

@bjornfor please try on latest master after #80369

Unfortunately:

$ git pull      # updated to 69efb5cd343a947502bf2372c0385e0afc84c570
$ nix-build -A neovim
...
-- Installing: /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lhandle.h
-- Installing: /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lreq.h
shrinking RPATHs of ELF executables and libraries in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
shrinking /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so
strip is /nix/store/xpzdkp9y8fnliwrs2g5pygx2qmmbrv8w-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib
patching script interpreter paths in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
checking for references to /build/ in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1...
cp: cannot stat '/nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1': No such file or directory

Error: Failed copying /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1 to /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/libluv.so.1
builder for '/nix/store/r1r1sprc1r1yb29w17vfyfnmi2r09zjv-luajit-2.1.0-beta3-luv-1.34.1-1.drv' failed with exit code 1
cannot build derivation '/nix/store/0xasdnwd3fvxzl7qrs2zf5wsh2f86yrj-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6v9iv9jrlwik9a3m5v4hn6gwhph1sc27-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/6v9iv9jrlwik9a3m5v4hn6gwhph1sc27-neovim-0.4.3.drv' failed

I looked inside the store path it's trying to copy from, there is not a single *.so file there:

$ find /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/ -ls
 12867484      4 drwxr-xr-x   4 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/
 12867706      4 drwxr-xr-x   3 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include
 12867707      4 drwxr-xr-x   2 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv
 12867710      8 -rw-r--r--   1 nixbld1  nixbld       5678 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/luv.h
 12867713      4 -rw-r--r--   1 nixbld1  nixbld       1534 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lreq.h
 12867711      4 -rw-r--r--   1 nixbld1  nixbld       1744 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/util.h
 12867712      4 -rw-r--r--   1 nixbld1  nixbld       2225 Jan 18 06:04 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lhandle.h
 12867491      4 drwxr-xr-x   2 nixbld1  nixbld       4096 Feb 17 20:15 /nix/store/s7kkgp1i5nl383wkc392zwpb88fl03cr-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks
bjornfor picture bjornfor on 17 Feb 2020
👍1

I can confirm @bjornfor's findings, it's not working for me, too.

I can also find the missing library at /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so.

jokogr picture jokogr on 17 Feb 2020

I uhh… don't know what to tell you:

HEAD is now at eaa47284113 luaPackages.luv: 1.30.0-0 -> 1.34.1-1
hiro:nixpkgs:% nix-build --check -A luajitPackages.luv                                                                                                                                     <eaa47284113>
checking outputs of '/nix/store/vr6kd59vkzz0n2lrw6d3kssh4cqz1zvq-luajit-2.1.0-beta3-luv-1.34.1-1.drv'...
unpacking sources
unpacking source archive /nix/store/f0az73f8fi7fp7cz55iapzwr0wh1kd93-luv-1.34.1-1.src.rock

Done. You may now enter directory
luv-1.34.1-1/luv-1.34.1-1
and type 'luarocks make' to build.
source root is ./luv-1.34.1-1/luv-1.34.1-1
setting SOURCE_DATE_EPOCH to timestamp 1579323846 of file ./luv-1.34.1-1/luv-1.34.1-1/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.34.1-1 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/gk304dpcw0x4fx7vp3di9v1lv9s8rhdc-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/zhsgiagaqx27hp5j94hy6i46rv3gnh75-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.34.1-1/luv-1.34.1-1/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared module luv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/luv.so
No existing manifest. Attempting to rebuild...
luv 1.34.1-1 is now installed in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1 (license: Apache 2.0)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
shrinking /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/luv.so
strip is /nix/store/xpzdkp9y8fnliwrs2g5pygx2qmmbrv8w-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1/lib
patching script interpreter paths in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
checking for references to /build/ in /nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1...
/nix/store/rpdkdk7xifs4kylf6x5m5caijk71b7vf-luajit-2.1.0-beta3-luv-1.34.1-1
ArdaXi picture ArdaXi on 17 Feb 2020

I just installed neovim on a clean vm using the most recent nixpkgs tarball without errors...

mjlbach picture mjlbach on 17 Feb 2020

@ArdaXi could you try to build luajit.pkgs.luv.libluv as used there: https://github.com/NixOS/nixpkgs/blob/9992702e4beac0230600a014073c91e315227d2d/pkgs/applications/editors/neovim/default.nix#L47 (lua is overridden with luajit in all-packages.nix for neovim).

jokogr picture jokogr on 17 Feb 2020

@jokogr That one fails, but I'm not on Darwin so that's not a dependency of Neovim for me.

https://github.com/NixOS/nixpkgs/blob/9992702e4beac0230600a014073c91e315227d2d/pkgs/applications/editors/neovim/default.nix#L82-L84

Messy and strange indeed.

EDIT: I'm not entirely sure why a Darwin build would be trying to install .so files anyway…

ArdaXi picture ArdaXi on 18 Feb 2020

What _I_ don't get is what's the difference between libluv and luv - both packages seem to come from the same rockspec but it's unclear what's the difference between the two and why does nix-build -A luajit.pkgs.luv doesn't fail while nix-build -A luajit.pkgs.luv.libluv does.

doronbehar picture doronbehar on 18 Feb 2020

The difference between luajit.pkgs.luv and luajit.pkgs.luv.libluv:

sdi() { nix show-derivation "$(nix-instantiate -A "$1")" | sed -r 's,/nix/store/[^/]{32},/nix/store/...,g'; }
diff -U999 <(sdi luajit.pkgs.luv) <(sdi luajit.pkgs.luv.libluv)

Cleaned up a little bit to make it easier to read:

 {
     "inputDrvs": {
       ...
+      "/nix/store/...-hook.drv": [
+        "out"
+      ],
     },
     ...
     "env": {
+      "NIX_LDFLAGS": "",
       ...
-      "nativeBuildInputs": "",
+      "nativeBuildInputs": "/nix/store/...-hook",
       ...
-      "preBuild": "sed -i 's,\\(option(WITH_SHARED_LIBUV.*\\)OFF,\\1ON,' CMakeLists.txt\nrm -rf deps/libuv\n",
+      "preBuild": "sed -i 's,\\(option(WITH_SHARED_LIBUV.*\\)OFF,\\1ON,' CMakeLists.txt\nrm -rf deps/libuv\n
+                   sed -i 's,\\(option(BUILD_MODULE.*\\)ON,\\1OFF,' CMakeLists.txt\n
+                   sed -i 's,\\(option(BUILD_SHARED_LIBS.*\\)OFF,\\1ON,' CMakeLists.txt\n
+                   sed -i 's,${INSTALL_INC_DIR},/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9/include/luv,' CMakeLists.txt\n",
       ...
     }
   }
 }

(I'm unable to build neovim on x86_64-linux on master.)

chkno picture chkno on 19 Feb 2020

neovim on x86_64-linux on master works again since staging-merge 9d88ee08f6bf8a51f99872aedefd4139635f24ea.

chkno picture chkno on 21 Feb 2020
🎉2

And it builds on release-20.03 (tested 3b9b66d265d589946a20f04ea27490d7917d20db) too. Thanks!

bjornfor picture bjornfor on 21 Feb 2020

it builds on release because it doesn't contain d4f09b65906c1077397da825671f8b35e0175fdd, master still seems faulty https://github.com/NixOS/nixpkgs/issues/81206 ?

teto picture teto on 28 Feb 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

sid-kap picture sid-kap  Â·  3Comments
domenkozar picture domenkozar  Â·  3Comments
lverns picture lverns  Â·  3Comments
retrry picture retrry  Â·  3Comments
tomberek picture tomberek  Â·  3Comments