Nixpkgs: Patching packages that ship a configure script requires significant overrides

[nh2]

Issue description

When we use a pre-generated configure script shipped with some release tarball of upstream software, some nix things that we expect to work, like adding patches, no longer work.

For example, I tried to apply a patch to util-linux:

{
  pkgs.utillinux.overrideAttrs (oldAttrs: {
    patches = oldAttrs.patches ++ [
      # TODO: Remove when util-linux v2.32 is available to us.
      ./extra-nixpkgs/util-linux-unshare-Add-kill-child-option-squashed-v2.31.patch
    ];
  }
}

Turns out this doesn't work.

When applying a patch that adds a new file, you can no longer use the configure script delivered with the package; you need to regenerate it with autoconf/automake.

For example, I got this output:

build flags: -j8 -l8 SHELL=/nix/store/hvhmhd625l47wcvxrachk255j02nhkkp-bash-4.4-p12/bin/bash usrbin_execdir=\$\(bin\)/bin usrsbin_execdir=\$\(bin\)/sbin
 cd . && /nix/store/hvhmhd625l47wcvxrachk255j02nhkkp-bash-4.4-p12/bin/bash /tmp/nix-build-util-linux-2.31.drv-0/util-linux-2.31/config/missing automake-1.15 --foreign Makefile
configure.ac:14: error: version mismatch.  This is Automake 1.15.1,
configure.ac:14: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:14: comes from Automake 1.15.  You should recreate
configure.ac:14: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.15' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'automake' program is part of the GNU Automake package:
         <http://www.gnu.org/software/automake>
         It also requires GNU Autoconf, GNU m4 and Perl in order to run:
         <http://www.gnu.org/software/autoconf>
         <http://www.gnu.org/software/m4/>
         <http://www.perl.org/>
make: *** [Makefile:5009: Makefile.in] Error 63
builder for ‘/nix/store/yn49k3bvks61b6lgi26306cg6zx9fgvb-util-linux-2.31.drv’ failed with exit code 2

To get the build working with patches, I had to depend on util-linux's _actual_ build dependencies instead of relying on the pre-generated configure file shipped with the release tarball:

{
  pkgs.utillinux.overrideAttrs (oldAttrs: {
    patches = oldAttrs.patches ++ [
      # TODO: Remove when util-linux v2.32 is available to us.
      ./extra-nixpkgs/util-linux-unshare-Add-kill-child-option-squashed-v2.31.patch
    ];
    # For applying patches to work, we have to depend on autoconf/automake ,
    # and run `./autogen.sh` to regenerate the `configure` script.
    nativeBuildInputs = oldAttrs.nativeBuildInputs ++ (with pkgs; [ autoconf automake bison libtool gettext ]);
    postPatch = oldAttrs.postPatch + ''
      ./autogen.sh
    '';
  })
}

Steps to reproduce

For an example, try to patch util-linux like I do above, with this patch.

Technical details

• Nixpkgs version: 0f9ab62873e0d7c0adaaa5234ffd70cd3a42ba18 (on release-17.09)

[nh2]

Maybe our policy for making upstream packages should be that we always use the "full" build toolchain (e.g. running autoconf ourselves instead of relying on the upstream-pre-generated ./configure)?

I believe this would also help when overriding the src from an upstream tarball to an upstream git repository and have things still work, as upstream git repositories often don't carry pre-generated ./configure scripts.

[yegortimoshenko]

Nix is a source-based package manager, so of course we should build everything from source, including configure scripts and build systems.

[FRidh]

@nh2 have you tried adding the autoreconfHook instead?

[doronbehar]

Also, when considering automatic version updates mechanisms, using the original Git repository and autoreconf should ease bots which check for updates through in Git repositories vs release tarballs.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.