Nixpkgs: pkgsMusl.libffi fails in test suite

[basvandijk]

Describe the bug

On the latest nixpkgs master (8cc8b3b235368fad651229cfe9037cd236982f70):

$ nix-build -A pkgsMusl.libffi
these derivations will be built:
  /nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv
building '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv'...
...
running tests
check flags: SHELL=/nix/store/sydxyv59z4dka8h22if5v5p11nl1sf4w-bash-4.4-p23/bin/bash VERBOSE=y check
MAKE x86_64-pc-linux-musl : 0 * check
make[1]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl'
Making check in include
make[2]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/include'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/include'
Making check in testsuite
make[2]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make  check-DEJAGNU
make[3]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
Making a new site.exp file ...
srcdir='../../testsuite'; export srcdir; \
EXPECT=expect; export EXPECT; \
if /nix/store/sydxyv59z4dka8h22if5v5p11nl1sf4w-bash-4.4-p23/bin/bash -c "runtest --version" > /dev/null 2>&1; then \
  exit_status=0; l='libffi'; for tool in $l; do \
    if runtest --tool $tool --srcdir $srcdir  ; \
    then :; else exit_status=1; fi; \
  done; \
else echo "WARNING: could not find 'runtest'" 1>&2; :;\
fi; \
exit $exit_status
WARNING: Couldn't find the global config file.
Using ../../testsuite/lib/libffi.exp as tool init file.
Test run by nixbld on Tue Feb 11 20:53:11 2020
Target is .runtest-wrapped
Host   is x86_64-pc-linux-musl

                === libffi tests ===

Schedule of variations:
    unix

Running target unix
Using /nix/store/ca489ppzi5p23dv9bk5l3dmcjsd4sasr-dejagnu-1.6.2/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /nix/store/ca489ppzi5p23dv9bk5l3dmcjsd4sasr-dejagnu-1.6.2/share/dejagnu/config/unix.exp as generic interface file for target.
Using ../../testsuite/config/default.exp as tool-and-target-specific interface file.
WARNING: Assuming target board is the local machine (which is probably wrong).
You may need to set your DEJAGNU environment variable.
Running ../../testsuite/libffi.bhaible/bhaible.exp ...
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=1 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O0 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=1 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O2 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=2 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O0 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=2 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O2 (test for excess errors)
...
Running ../../testsuite/libffi.call/call.exp ...
FAIL: libffi.call/align_mixed.c -W -Wall -Wno-psabi -O0 (test for excess errors)
FAIL: libffi.call/align_mixed.c -W -Wall -Wno-psabi -O2 (test for excess errors)
FAIL: libffi.call/align_stdcall.c -W -Wall -Wno-psabi -O0 (test for excess errors)
...
Running ../../testsuite/libffi.closures/closure.exp ...
Running ../../testsuite/libffi.complex/complex.exp ...
Running ../../testsuite/libffi.go/go.exp ...

                === libffi Summary ===

# of expected passes            280
# of unexpected failures        264
# of unsupported tests          206
make[3]: *** [Makefile:449: check-DEJAGNU] Error 1
make[3]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make[2]: *** [Makefile:526: check-am] Error 2
make[2]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make[1]: *** [Makefile:1347: check-recursive] Error 1
make[1]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl'
make: *** [Makefile:2949: check] Error 2
builder for '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv' failed with exit code 2
error: build of '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv' failed

Metadata

 nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.98, NixOS, 19.09pre-git (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.2`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: libffi

[basvandijk]

@nh2 seen this before by any chance?

[basvandijk]

Note that pkgsMusl.libffi builds successfully with 82875a20ba444110396f95537b18247898e40e22.

[basvandijk]

git bisect comes up with 2476d6decabf1be63151dd94be15e87fad52fb0b as the first bad commit:

$ git bisect log
git bisect start
# bad: [8cc8b3b235368fad651229cfe9037cd236982f70] Merge pull request #73777 from doronbehar/package-brotab
git bisect bad 8cc8b3b235368fad651229cfe9037cd236982f70
# good: [82875a20ba444110396f95537b18247898e40e22] wasm-bindgen-cli: 0.2.57 -> 0.2.58
git bisect good 82875a20ba444110396f95537b18247898e40e22
# good: [cd27940df06d39353bd47c2cea096e9c8c026591] Merge pull request #78188 from andir/buildRustCrate-support-rlib
git bisect good cd27940df06d39353bd47c2cea096e9c8c026591
# good: [f4f39fc8248ea89f50c08259154f48a7ceda9216] Merge pull request #79309 from jtojnar/hugsie-pkgs
git bisect good f4f39fc8248ea89f50c08259154f48a7ceda9216
# bad: [8847d57763ec4bdb71264fc0f19b24901a345705] Merge staging-next into staging
git bisect bad 8847d57763ec4bdb71264fc0f19b24901a345705
# good: [8396961c9cc67d0a12c0bcf475de77d9a849fd7b] Merge pull request #79300 from jtojnar/default-wm-fix
git bisect good 8396961c9cc67d0a12c0bcf475de77d9a849fd7b
# good: [ed095a6965af671e8478a56446d77f4a4591521c] Merge pull request #74666 from r-ryantm/auto-update/sysstat
git bisect good ed095a6965af671e8478a56446d77f4a4591521c
# bad: [feaf32c35337562c26b48f2f8f6a7f16e5386eca] gmp: 6.1.2 -> 6.2.0
git bisect bad feaf32c35337562c26b48f2f8f6a7f16e5386eca
# bad: [c24fc44e1f1cd7fb4b4f25d174f9284b0d85a154] jfbview: fix build w/glibc-2.30
git bisect bad c24fc44e1f1cd7fb4b4f25d174f9284b0d85a154
# bad: [8e7bbb7d4f70957f1bb7ed049cb038236be58bf7] postfix: fix build w/glibc 2.30
git bisect bad 8e7bbb7d4f70957f1bb7ed049cb038236be58bf7
# good: [c17058226ae769e3666949564904fa71beb5ded6] libunistring: disable tests
git bisect good c17058226ae769e3666949564904fa71beb5ded6
# bad: [2d36eac7801824d5b6d7cc43958b9733e5901109] stdenv: rebuild aarch64 bootstrap files with gcc8
git bisect bad 2d36eac7801824d5b6d7cc43958b9733e5901109
# good: [01ce36f7883cd8fbbeeb3d023222ef031228f6bd] bzip2: fix aarch64 build
git bisect good 01ce36f7883cd8fbbeeb3d023222ef031228f6bd
# bad: [2476d6decabf1be63151dd94be15e87fad52fb0b] expect: fix aarch64 build
git bisect bad 2476d6decabf1be63151dd94be15e87fad52fb0b
# first bad commit: [2476d6decabf1be63151dd94be15e87fad52fb0b] expect: fix aarch64 build

[basvandijk]

@Ma27 any idea why https://github.com/NixOS/nixpkgs/commit/2476d6decabf1be63151dd94be15e87fad52fb0b breaks pkgsMusl.libffi?

[basvandijk]

Note that removing the autoreconfHook from expect's nativeBuildInputs prevents the pkgsMusl.libffi test failure.

[Ma27]

Oof, that sounds rather nasty :(
Currently building the packages on the community builder, will try to investigate what's going on there today :)

[basvandijk]

I compared two builds of expect. One with autoreconfHook and one without.

When running without autoreconfHook I see the following during configuring:

checking stty to use... /nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty

When running with autoreconfHook I see it setting the wrong path to stty:

checking stty to use... /bin/stty

Also when I compare the configure scripts between the builds I see the following diff:

• nix-build-expect-5.45.4.drv-0: without autoreconfHook
• nix-build-expect-5.45.4.drv-1: with autoreconfHook

--- /tmp/nix-build-expect-5.45.4.drv-0/expect5.45.4/configure   2020-02-12 14:58:13.913396295 +0000
+++ /tmp/nix-build-expect-5.45.4.drv-1/expect5.45.4/configure   2020-02-12 14:58:52.129964483 +0000
@@ -730,7 +730,6 @@
 docdir
 oldincludedir
 includedir
-runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -811,7 +810,6 @@
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
-runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
@@ -1064,15 +1062,6 @@
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;

-  -runstatedir | --runstatedir | --runstatedi | --runstated \
-  | --runstate | --runstat | --runsta | --runst | --runs \
-  | --run | --ru | --r)
-    ac_prev=runstatedir ;;
-  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
-  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
-  | --run=* | --ru=* | --r=*)
-    runstatedir=$ac_optarg ;;
-
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1210,7 +1199,7 @@
 for ac_var in  exec_prefix prefix bindir sbindir libexecdir datarootdir \
        datadir sysconfdir sharedstatedir localstatedir includedir \
        oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-       libdir localedir mandir runstatedir
+       libdir localedir mandir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1363,7 +1352,6 @@
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
-  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -8639,10 +8627,10 @@

 { $as_echo "$as_me:${as_lineno-$LINENO}: checking stty to use" >&5
 $as_echo_n "checking stty to use... " >&6; }
-if test -r /usr/local/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty ; then
-  STTY_BIN=/usr/local/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty
+if test -r /usr/local/bin/stty ; then
+  STTY_BIN=/usr/local/bin/stty
 else
-  STTY_BIN=/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty
+  STTY_BIN=/bin/stty
 fi
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STTY_BIN" >&5
 $as_echo "$STTY_BIN" >&6; }

Why would autoreconfHook cause an invalid path to stty?

[Ma27]

Why would autoreconfHook cause an invalid path to stty?

No idea tbh.... But the build was pretty weird before - I still don't exactly know why the addition was needed.

Does expect currently build fine on aarch64 without the autoreconfHook btw?

[basvandijk]

I'll make a PR out of https://github.com/dfinity-lab/nixpkgs/commit/a4e23f32f8d7006e217125a9123b53597b9c0c24 and ask ofborg to build expect on aarch64.

[nh2]

@nh2 seen this before by any chance?

Indeed, my CI noticed this failure, the first failing build is https://buildkite.com/nh2/static-haskell-nix/builds/287#52588b27-84ce-4cd7-97ee-4496fcaa2e37

[nh2]

I'm building static-haskell-nix on top of your patch here:

https://buildkite.com/nh2/static-haskell-nix/builds/291#8eed5f13-bcf1-48c8-b084-600150bad27b

Also noticed that another thing broke though:

• In that build, apparently I can no longer previous.fetchurl.override, it says value is a function while a set was expected.

[nh2]

* In that build, apparently I can no longer `previous.fetchurl.override`, it says `value is a function while a set was expected`.

My bad, I ran the wrong commit.

This is the real build: https://buildkite.com/nh2/static-haskell-nix/builds/294

80006 does seem to fix the problem; my working set already succeeded ~(other package sets still building)~ all green.