Nixpkgs: python2Packages.nixpart0 doesn't build

[kolloch]

Describe the bug

python2Packages.nixpart0 fails to build.

Tested on both

• current nixpkgs (commit 01790228caaeb9ac8e16bb9ef32a925592159248)
• nixpkgs 20.03

Impact: It is apparently used as part of nixops deploy on hetzner, so my deployment is blocked on this.

To Reproduce
Steps to reproduce the behavior:

nix build -L -f '<nixpkgs>' python2Packages.nixpart0
or
nix build -L -f . python2Packages.nixpart0

in the checked out nixpkgs sources.

Expected behavior

The package builds without errors.

Screenshots

The end of the build output:

  /nix/store/kgp3vq8l9yb8mzghbw83kyr3f26yqvsz-bash-4.4-p23/bin/bash ../libtool  --tag=CC   --mode=link gcc  -Wall -g -O2   -o cryptsetup cryptsetup-utils_crypt.o cryptsetup-utils_loop.o cryptsetup-utils_fips.o cryptsetup-utils_tools.o cryptsetup-utils_password.o cryptsetup-cryptsetup.o ../lib/libcryptsetup.la -lpopt  
  libtool: link: gcc -Wall -g -O2 -o .libs/cryptsetup cryptsetup-utils_crypt.o cryptsetup-utils_loop.o cryptsetup-utils_fips.o cryptsetup-utils_tools.o cryptsetup-utils_password.o cryptsetup-cryptsetup.o  ../lib/.libs/libcryptsetup.so -lpopt -Wl,-rpath -Wl,/nix/store/gdyd8rvq0982axbhfabjikqlgvih7nqp-cryptsetup-1.6.3/lib
  /nix/store/3b3ighb83nhifa1v4n7855hlbdl1mhf9-binutils-2.31.1/bin/ld: cryptsetup-utils_loop.o: in function `_sysfs_backing_file':
  /build/cryptsetup-1.6.3/src/../lib/utils_loop.c:192: undefined reference to `minor'
  /nix/store/3b3ighb83nhifa1v4n7855hlbdl1mhf9-binutils-2.31.1/bin/ld: /build/cryptsetup-1.6.3/src/../lib/utils_loop.c:192: undefined reference to `major'
  /nix/store/3b3ighb83nhifa1v4n7855hlbdl1mhf9-binutils-2.31.1/bin/ld: cryptsetup-utils_loop.o: in function `crypt_loop_device':
  /build/cryptsetup-1.6.3/src/../lib/utils_loop.c:222: undefined reference to `major'
  /nix/store/3b3ighb83nhifa1v4n7855hlbdl1mhf9-binutils-2.31.1/bin/ld: ../lib/.libs/libcryptsetup.so: undefined reference to `makedev'
  collect2: error: ld returned 1 exit status
  make[2]: *** [Makefile:613: cryptsetup] Error 1
  make[2]: Leaving directory '/build/cryptsetup-1.6.3/src'
  make[1]: *** [Makefile:451: all-recursive] Error 1
  make[1]: Leaving directory '/build/cryptsetup-1.6.3'
  make: *** [Makefile:382: all] Error 2
cannot build derivation '/nix/store/45lhiz89i33x1w1sy8i1q7qk5xrrahn8-blivet-0.17-1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/c36z6ivydb5wpp6zzban6vd401mwk1yg-nixpart-0.4.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/c36z6ivydb5wpp6zzban6vd401mwk1yg-nixpart-0.4.1.drv' failed

Here is the full build output.

Additional context

I found ancient cryptsetup mailing list references to this build problem. I tried to patch in sysmacro includes to fix the problem without avail.

The URL used in the cryptsetup source, doesn't work anymore. I used the gitlab sources but they don't include the configure script yet, so I needed additional buildInputs to run "autogen.sh".

These are my attempts:
https://github.com/kolloch/nixpkgs/commit/60486dbcae0aedf4f56f5e11735e85e40610f187

Notify maintainers
@aszlig

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.0-26-generic, Ubuntu, 20.04 LTS (Focal Fossa)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`
 - channels(root): `""`
 - channels(peter): `"home-manager, nixpkgs-20.03.1619.ab3adfe1c76"`
 - nixpkgs: `/home/peter/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: [ "python2Packages.nixpart0" ]
# a list of nixos modules affected by the problem
module:

[kolloch]

Oh, and there are tests for this in nixos/tests/partition.nix but they are apparently not run. Or are they?

[mkg20001]

There's now a 100 USD bounty on this issue: https://www.bountysource.com/issues/91403976-fresh-nixops-deploy-on-hetzner-fails-because-of-broken-build2packages-nixpart0

[pbogdan]

FWIW I've collected build fixes in:

• https://github.com/pbogdan/nixpkgs/tree/nixpart0 (for master)
• https://github.com/pbogdan/nixpkgs/tree/nixpart0-2003 (for 20.03)

Unfortunately the more I started digging into this the less confidence I have in any of this actually working at all… If it's used for relatively simple bootstraping then it _might_ be OK. I'm not however going to spend more time working on this but perhaps the fixes above might be of some use to someone.

[kolloch]

If this ancient stuff is hard to fix, the dependent code should probably be based on the newer nixpart.

[nh2]

Impact: It is apparently used as part of nixops deploy on hetzner, so my deployment is blocked on this.

Same here.

[nh2]

For easier finding with Google for others:

This happens on 20.03 when nixops tries to build nix/hetzner-bootstrap.nix when formatting Hetzner disks via the Hetzner rescue system.

The derivations it's going to try to build are:

lvm2-2.02.106.drv
dmraid-1.0.0.rc15.drv
parted-3.1.drv
pyparted-3.10.drv
cryptsetup-1.6.3.drv
multipath-tools-0.4.9.drv
pyblock-0.53.drv
blivet-0.17-1.drv
nixpart-0.4.1.drv
hetzner-nixops-installer.drv

[nh2]

There's now a 100 USD bounty

Make that 200 USD.

I'm not however going to spend more time working on this but perhaps the fixes above might be of some use to someone.

@pbogdan This worked for me, with your patches I could use nixops again and partition our Hetzner server. If you're happy to PR that into nixpkgs (with backport to 20.03), I'd count that as bounty fulfillment.

Alternatively, making nixops+hetzner it work with newer nixpart would also do it.

[pbogdan]

I don't feel qualified to try and port to newer nixpart but would be glad to open a PR with my fixes. And perhaps with the added visibility of the PR someone will come up with something better?

I should be able to get around to it either today evening or tomorrow, will CC you in when I open it and if / when the PR is accepted I can prepare a backport as well.

[aszlig]

Notify maintainers
@aszlig

Damn, I missed that notification and just stumbled on this by myself, sorry everyone :-(

@pbogdan: Did you test whether you could bootstrap a Hetzner machine with your fixes?

[nh2]

There's now a 100 USD bounty on this issue: https://www.bountysource.com/issues/91403976-fresh-nixops-deploy-on-hetzner-fails-because-of-broken-build2packages-nixpart0

Thanks again for fixing this! I consider the bounty as fulfilled, please go ahead @pbogdan to claim it :+1:

[nh2]

@pbogdan I think you need to click Solve problem on the bountysource page and then we should get a prompt.

[pbogdan]

Hmm, I think that unless https://github.com/NixOS/nixops-hetzner/issues/36 is closed (which the issue associated with the bounty instead of this one) I won't be able to submit a claim. But can give it a try and see what happens.

[pbogdan]

Indeed, I got a message saying:

You will be able to claim this bounty as soon as the original issue is marked as closed.

[mkg20001]

Maybe it will work if the issue gets re-opened and then closed again? (//cc people who could do that)

[pbogdan]

I think we would either need the OP or someone who can manage issues over at nixops-hetzner to close out the issue over there. Until then I think we are kinda stuck.

[pbogdan]

@nh2 @mkg20001 Would you have any objections if I asked Bountysource to lift the restriction that requires https://github.com/NixOS/nixops-hetzner/issues/36 to be closed before the bounty can be claimed? They indicated it's a possibility when I asked what's the best way forward but I'm not sure if that would be appropriate. The backers would still be able to reject / dispute the claim. What do you think?

PS. Sorry for hijacking this issue somewhat.

[mkg20001]

No objections, go ahead

[nh2]

Same here, no objections.

[pbogdan]

Just a heads up - BS relinked it to this issue (https://www.bountysource.com/issues/91458370-python2packages-nixpart0-doesn-t-build) and I was able to open a claim.