Nixpkgs: neovim install fails with missing libluv.so.1

[mweinelt]

Describe the bug
neovim fails to build with missing libluv.so.1, similar to #80704 #80750, on b1ec189c9faa017e7a8dc3aa3b8243da3d61b8c7.

building '/nix/store/nv873vikr2nrm02f0lmsicp1lz49yg85-luajit-2.1.0-beta3-luv-1.34.1-1.drv' on 'ssh://[email protected]'...
unpacking sources
unpacking source archive /nix/store/f0az73f8fi7fp7cz55iapzwr0wh1kd93-luv-1.34.1-1.src.rock

Done. You may now enter directory
luv-1.34.1-1/luv-1.34.1-1
and type 'luarocks make' to build.
source root is ./luv-1.34.1-1/luv-1.34.1-1
setting SOURCE_DATE_EPOCH to timestamp 1579323846 of file ./luv-1.34.1-1/luv-1.34.1-1/src/work.c
patching sources
configuring
building
installing

luv 1.34.1-1 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/1kn7fi3hhi33jms3113riyzwyn2yqpqd-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/1kn7fi3hhi33jms3113riyzwyn2yqpqd-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/1kn7fi3hhi33jms3113riyzwyn2yqpqd-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
checking for references to /build/ in /nix/store/zfbnqbpirc2hww8v9bw32k50zh2dn02y-tlp-1.3.1...
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/w89ndqsb7ig22mw5hk7k7zfvxmhx44xb-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/1m8vh1byjnjmjkpdidcqc37cykgsqjap-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/1m8vh1byjnjmjkpdidcqc37cykgsqjap-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.34.1-1/luv-1.34.1-1/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
building '/nix/store/vjbx09i2wqqa5l4ca1xi1azdps0jdbgc-system-generators.drv'...
building '/nix/store/4qlfrm3s3frdkmcbc6a6kjwrvj925f19-system-path.drv'...
[100%] Linking C shared library libluv.so
[100%] Built target luv
[100%] Built target luv
created 16964 symlinks in user environment
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1.34.1
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/pkgconfig/libluv.pc
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/luv.h
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/util.h
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lhandle.h
-- Installing: /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/include/luv/lreq.h
cp: cannot stat '/nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1': No such file or directory

Error: Failed copying /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/luv-1.34.1-1-rocks/luv/1.34.1-1/lib/libluv.so.1 to /nix/store/6xqkgnj1hjgbg3jvxvzl74g924vw1lzj-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/libluv.so.1
builder for '/nix/store/nv873vikr2nrm02f0lmsicp1lz49yg85-luajit-2.1.0-beta3-luv-1.34.1-1.drv' failed with exit code 1
error: build of '/nix/store/nv873vikr2nrm02f0lmsicp1lz49yg85-luajit-2.1.0-beta3-luv-1.34.1-1.drv' on 'ssh://[email protected]' failed: builder for '/nix/store/nv873vikr2nrm02f0lmsicp1lz49yg85-luajit-2.1.0-beta3-luv-1.34.1-1.drv' failed with exit code 1
builder for '/nix/store/nv873vikr2nrm02f0lmsicp1lz49yg85-luajit-2.1.0-beta3-luv-1.34.1-1.drv' failed with exit code 1
cannot build derivation '/nix/store/c671xz1gwswnbsp8yk6x6npy30n6wih3-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6ildjqw91xp0fd7w7s91qxykx7d0h3hp-neovim-0.4.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yzklv88xsxglbcl9iw7jbm2825y998vb-home-manager-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/0zza1w7na8178bsqdl3y4hjrghy7nrlp-home-manager-generation.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8h1mqmxrv2p3wlwlrrwd0zi3ydj54j28-activate-hexa.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2h50s6f89rwzl1pphjck3a8qby2vcpmk-unit-home-manager-hexa.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9sdyrkf2nq87izpy39ymy4yqd9v3rj2b-system-units.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/m9bxp3y07h6skxd3p6zdcryb9cx9kwwl-etc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6z8kzrzmi0hrjfn8kwavgv7gb9c3pmrr-nixos-system-nyx-20.09.git.b1ec189c9fa.drv': 1 dependencies couldn't be built
error: build of '/nix/store/6z8kzrzmi0hrjfn8kwavgv7gb9c3pmrr-nixos-system-nyx-20.09.git.b1ec189c9fa.drv' failed

To Reproduce
Steps to reproduce the behavior:

1. add neovim to environment.systemPackages
2. rebuild from master (b1ec189c9faa017e7a8dc3aa3b8243da3d61b8c7)

Expected behavior
The package should successfully build.

Metadata
On master (b1ec189c9faa017e7a8dc3aa3b8243da3d61b8c7)

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: neovim

[mweinelt]

nothing to do with this commit; neovim is broken on master; we are working on a fix. Meanwhile revert the latest luarocks bump and it will work again.

via @teto in https://github.com/NixOS/nixpkgs/commit/7aee5b838b25a353a453aa7db0560390dc8089cf#commitcomment-37528069

[teto]

so you could do git revert -m 1 0566b5ce19fabe3c70ab1f56415dd8a019fa0aa8 until a proper fix.

[charvp]

Reverting the luarocks bump doesn't work for me. That fix is also very surprising to me, since neovim broke (for me) after the latest nixos-unstable channel update.

[cole-h]

Interesting. It worked fine for me. Running the above revert command and then home-manager switch -I nixpkgs=. (inside my local nixpkgs with it reverted) let me build neovim properly.

I don't think it's particularly surprising if updating a channel that happens to include the luarocks bump causes it to break, unless I'm misunderstanding you.

[charvp]

I updated from a channel where the luarocks bump had already happened, and neovim worked on that channel. In fact, I reverted to the previous hydra evaluation to make my system evaluate again.

[mdedetrich]

@cole-h This was actually before and it was working without problems and then suddenly the problem appeared again.

I can also report that I cam getting this in the same way @charvp described (updated latest nixos-unstable channel and then neovim stopped building again).

[vcunat]

I confirm this fixed by #81483 on master (channels may take a while). For now I'll assume that charvp's failure to resolve the issue with that revert was some kind of mistake; we can re-open otherwise, perhaps separately if it's a different kind of error.

[mdedetrich]

Does anyone know why updating the luarocks dependency is so brittle? Is it possible to at least add a test so we don't regress again in the future?

[teto]

@mdedetrich the tool exists (nixpkgs-review) but it takes CPU and disk space to run plus some reviewer time (and was not run in this case).
Also for this specific case, the luarocks update broke in subtle ways the build so I wouldn't call it brittle.

It is called unstable for a reason. The good thing about nixpkgs is you can pin neovim to an older version while keeping your system on unstable.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/best-practice-for-pinning-version-of-individual-packages/6194/1

[rpearce]

Are we sure this is fixed on Darwin? https://hydra.nixos.org/job/nixpkgs/trunk/neovim-unwrapped.x86_64-darwin still shows failing with logs of:

[100%] Building C object src/nvim/CMakeFiles/nvim.dir/window.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xdiffi.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xemit.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xhistogram.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xpatience.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xprepare.c.o
[100%] Building C object src/nvim/CMakeFiles/nvim.dir/xdiff/xutils.c.o
make[2]: *** No rule to make target '/nix/store/nyf0qhx47vad558rcv8bskfkdvs9wwav-luajit-2.1.0-beta3-luv-1.34.1-1/lib/lua/5.1/libluv.dylib', needed by 'bin/nvim'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:5012: src/nvim/CMakeFiles/nvim.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/bpxyw4r1h2wklcwc6kiisq2n9cps67k9-neovim-unwrapped-0.4.3.drv' failed with exit code 2

Or is this a different issue?

[thefloweringash]

I think the root problem here is that luarocks has a poor implementation of "move". Combined with the build's behavior being affected by filesystem ordering the result is essentially random.

The implementation of move calls out to the operating system commands [cp][] and [rm][]. Building with NIX_DEBUG = 2 will contain a sequence of cp, chmod and rm. I've extracted the relevant part of the log. The critical aspect here is that by default, unix cp will dereference symlinks and copy their contents.

My theory: luarocks scans the directory for external libraries in filesystem order
and attempts to move all shared libraries from /luv-1.34.1-1-rocks/luv/1.34.1-1/lib/ to /lib/lua/5.1/. At which point one of 3! things will happen, but the two main ones are:

Failure case

1. libluv.1.34.1.dylib is copied and deleted
2. libluv.dylib and libluv.1.dylib, now broken symlinks, fail to copy

Success Case

1. libluv.dylib and libluv.1.dylib are copied by dereferencing, and deleted
2. libluv.1.34.1.dylib is copied and deleted

Either way is wrong to some degree, but one of them happens to work well enough for downstream packages to build.

If I build locally, the filesystem order I get seems to work, so we can inspect the two results:

Failure case

$ tree -sh ./result/lib/lua/5.1 ./result/luv-1.34.1-1-rocks/luv/1.34.1-1/lib
./result/lib/lua/5.1
├── [136K]  libluv.1.34.1.dylib
└── [  96]  pkgconfig
    └── [ 436]  libluv.pc
./result/luv-1.34.1-1-rocks/luv/1.34.1-1/lib
├── [  19]  libluv.1.dylib -> libluv.1.34.1.dylib
└── [  14]  libluv.dylib -> libluv.1.dylib

Success case

$ tree -sh ./result/lib/lua/5.1 ./result/luv-1.34.1-1-rocks/luv/1.34.1-1/lib
./result/lib/lua/5.1
├── [136K]  libluv.1.34.1.dylib
├── [136K]  libluv.1.dylib
├── [136K]  libluv.dylib
└── [  96]  pkgconfig
    └── [ 436]  libluv.pc
./result/luv-1.34.1-1-rocks/luv/1.34.1-1/lib [error opening dir]

I've been focusing on this for darwin since the failure case happens to be cached, but I believe it applies equally to linux.

[teto]

@hishamhm is that sthg you have already witnessed ?

[hishamhm]

What LuaRocks version is this happening with? I have pushed fixes related to the file deployment stage in 3.3.1.

[thefloweringash]

I've tested 3.2.1 (current nixpkgs master) and 3.3.1 (hasty upgrade locally), they both have similar results, except on 3.3.1 the errors now propagate correctly. I've uploaded a log of a failed build with 3.3.1.

[thefloweringash]

I was hoping to find time to write a proper issue report and a test lua package for this, but haven't yet. My test case would have a directory structure made with touch a; ln -s a b, that looks like:

/tmp/luarocks-test
├── a
└── b -> a

In order to succeed the deployment stage would have to produce both the file and its symlink.