Nixpkgs: Add support for PEP 513 "manylinux1" wheels

[jonringer]

Issue description

I would like to add support for manylinux1 wheels as a build format.
Through the use of hooks, I would do an autoPatchelf stage on the .so's where they would get patched to linux against the corresponding dependencies:

libpanelw.so.5
libncursesw.so.5
libgcc_s.so.1
libstdc++.so.6
libm.so.6
libdl.so.2
librt.so.1
libc.so.6
libnsl.so.1
libutil.so.1
libpthread.so.0
libresolv.so.2
libX11.so.6
libXext.so.6
libXrender.so.1
libICE.so.6
libSM.so.6
libGL.so.1
libgobject-2.0.so.0
libgthread-2.0.so.0
libglib-2.0.so.0

How would this look from a python package perspective?

  pname = ...;
  version = ...;
  format = "manylinux1";

or something to that degree, there seems to be a few manylinux related PEPs: 513 571 600

I would probably also have to update fetchPypi to allow for fetching of the format as well with python version info:

 fetchPypi {
   inherit pname version format;
   sha256 = ...;
   pythonVersion = python.version;
  };

Additional changes/challenges

Need someway to say that our python intepreter is conditionally able to support doing manylinux1 builds.

Proposed solution (First thing that comes to mind):
hook for preBuild:

  export NIX_PYTHON_MANYLINUX=1

hook for fixupPhase:

   nativeBuildInputs = nativeBuildInputs ++ [ autoPatchelfHook ];
   buildInputs = buildInputs ++ [ ... list of required libraries ... ];

changes to python interpreter:

  echo "import os; manylinux1_compatible = not os.environ.get('NIX_PYTHON_MANYLINUX')" >> $out/lib/${libPrefix}/_manylinux.py

cc @FRidh

[jonringer]

related issues/PRs : #18484 https://github.com/j5api/j5/issues/352 #55812

[FRidh]

So, there are two situations here:

1. Building manylinux1 wheels
2. Using manylinux1 wheels

The first is interesting for the wider Python community as we could provide reproducibly built wheels. The second...we already support universal wheels so I suppose this is a logical continuation.

In that case, I propose for 2) a separate function that takes a wheel, the manylinux1 deps from #55812 and potentially extra deps, and produces a new wheel. That wheel could then be consumed by buildPython* and the hooks we have.

[jonringer]

This is mainly aimed at using manylinux1 wheels, and there's a few projects that will only release the pre-built wheels and no source :(

[FRidh]

Packages and lists for manylinux https://github.com/NixOS/nixpkgs/pull/73866.

I am not sure what to do with the configuration of manylinux. Right now it is very strict, changing the configuration requires you to essentially rebuild Python. Claiming manylinux compatibility is in my opinion not correct. The environment variable seems like a useful middleground, however, we should then have variables for manylinux1, manylinux2010 and manylinux2014, which is maybe too much.

I'm playing now a bit with poetry2nix (cc @adisbladis). Claiming manylinux compatibility is very useful in order to be able to lock a file, even when it downloads "broken" wheels. Maybe we just should not protect users.

[gleber]

There is one manylinux1-compatible Python derivation available in the open-source:
https://github.com/digital-asset/daml/blob/05e691f55852fbb207f0e43cf23bb95b95866ba3/nix/tools/python36/default.nix

It even has tests:
https://github.com/digital-asset/daml/tree/05e691f55852fbb207f0e43cf23bb95b95866ba3/dev-env/tests/manylinux1

[FRidh]

_manylinux.py has been removed https://github.com/NixOS/nixpkgs/pull/75763

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/mach-nix-create-python-environments-quick-and-easy/6858/16