Pip: Install manylinux wheel although it's not officially supported by the host

[b52]

• Pip version: 8.1.2
• Python version: 3.5.2
• Operating System: alpine 3.4
  

  Description:

Support the installation of manylinux wheels, although the host system doesn't officially support it.

For example, alpine officially uses musl, but one can install glibc and all other required libs flawlessly, which should enable the usage of manylinux wheels.

[RonnyPfannschmidt]

uhm wouldn't the python on alpine by default break due to being linked to musl while the code is linked to glibc (i recall such issues due to glib symbol versioning)

also many linux support is enabled by default even on platforms that might not support it

so unless alpine opts out of it, pip will happily install such packages

[b52]

Well, if in start a simple python:3.5.2-alpine Docker image, which has done nothing to prevent the installation of manylinux wheels, as one can see in the Dockerfile), pip still tries to install the source package.

[dstufft]

manylinux1 support isn't globally enabled on all Linux instances. We do some checks to try and figure out if the platform is manylinux1 friendly or not (mostly related to checking glibc version) but those checks are not very strict. I think it's really just the glibc version check.

You can override our detection logic (assuming you're still on a Linux at all that is) by writing a _manylinux.py file somewhere on sys.path and adding a single variable to it:

# _manylinux.py

manylinux1_compatible = True  # or False if you want to disable it

I don't fully grok all of the ABI issues with libc, but I think the concern is not whether or not glibc is the "primary" libc but what version of libc Python itself is linked against. I don't know if that has to match what c-extensions are linked against or not.

Try the above though! It should allow installation of manylinux1 wheels, and if it seems to be working maybe we can explore better detection methods.

[b52]

@dstufft Thanks, that addition indeed allows the installation of wheel binaries.

[jbwdevries]

I'd like to report that, at least for typed_ast==1.3.1 (latest) on `alpine 3.9 (latest), the given method doesn't work. When importing, I get the following error:

ImportError: Error relocating /usr/lib/python3.6/site-packages/typed_ast/_ast3.cpython-36m-x86_64-linux-gnu.so: __rawmemchr: symbol not found

I couldn't really work around it, except for modifying python3 to run with the compatibility layer:

apk add gcompat patchelf
patchelf --set-interpreter /lib/ld-linux-x86-64.so.2 $(which python3)

[ihnorton]

It's not a great idea, but in a pinch you can also manually specify -t /path/to/dist-packages (or wherever), which will override the compatibility check entirely.

[stdedos]

FYI for future readers: In Alpine, one of the packages _definitely_ don't like one or more of these packets:

Requirement already satisfied: mako in ./venv-example/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (1.0.10)
Requirement already satisfied: cherrypy in ./venv-example/lib/python3.6/site-packages (from -r requirements.txt (line 2)) (18.1.1)
Requirement already satisfied: pysaml2<5.0.0,>=4.7.0 in ./venv-example/lib/python3.6/site-packages (from -r requirements.txt (line 3)) (4.7.0)
Requirement already satisfied: MarkupSafe>=0.9.2 in ./venv-example/lib/python3.6/site-packages (from mako->-r requirements.txt (line 1)) (1.1.1)
Requirement already satisfied: cheroot>=6.2.4 in ./venv-example/lib/python3.6/site-packages (from cherrypy->-r requirements.txt (line 2)) (6.5.5)
Requirement already satisfied: zc.lockfile in ./venv-example/lib/python3.6/site-packages (from cherrypy->-r requirements.txt (line 2)) (1.4)
Requirement already satisfied: more-itertools in ./venv-example/lib/python3.6/site-packages (from cherrypy->-r requirements.txt (line 2)) (7.0.0)
Requirement already satisfied: portend>=2.1.1 in ./venv-example/lib/python3.6/site-packages (from cherrypy->-r requirements.txt (line 2)) (2.4)
Requirement already satisfied: defusedxml in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (0.6.0)
Requirement already satisfied: cryptography>=1.4 in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2.6.1)
Requirement already satisfied: six in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (1.12.0)
Requirement already satisfied: pytz in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2019.1)
Requirement already satisfied: requests>=1.0.0 in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2.22.0)
Requirement already satisfied: pyOpenSSL in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (19.0.0)
Requirement already satisfied: python-dateutil in ./venv-example/lib/python3.6/site-packages (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2.8.0)
Requirement already satisfied: backports.functools-lru-cache in ./venv-example/lib/python3.6/site-packages (from cheroot>=6.2.4->cherrypy->-r requirements.txt (line 2)) (1.5)
Requirement already satisfied: setuptools in ./venv-example/lib/python3.6/site-packages (from zc.lockfile->cherrypy->-r requirements.txt (line 2)) (39.1.0)
Requirement already satisfied: tempora>=1.8 in ./venv-example/lib/python3.6/site-packages (from portend>=2.1.1->cherrypy->-r requirements.txt (line 2)) (1.14.1)
Requirement already satisfied: asn1crypto>=0.21.0 in ./venv-example/lib/python3.6/site-packages (from cryptography>=1.4->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (0.24.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in ./venv-example/lib/python3.6/site-packages (from cryptography>=1.4->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (1.12.3)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./venv-example/lib/python3.6/site-packages (from requests>=1.0.0->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (1.25.3)
Requirement already satisfied: certifi>=2017.4.17 in ./venv-example/lib/python3.6/site-packages (from requests>=1.0.0->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2019.3.9)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./venv-example/lib/python3.6/site-packages (from requests>=1.0.0->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in ./venv-example/lib/python3.6/site-packages (from requests>=1.0.0->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2.8)
Requirement already satisfied: jaraco.functools>=1.20 in ./venv-example/lib/python3.6/site-packages (from tempora>=1.8->portend>=2.1.1->cherrypy->-r requirements.txt (line 2)) (2.0)
Requirement already satisfied: pycparser in ./venv-example/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.4->pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3)) (2.19)

[Trollgeir]

Based on @dstufft I can confirm that this fixed it for my docker image running python:3.7-alpine for the packages I needed (catboost).

RUN echo 'manylinux1_compatible = True' > /usr/local/lib/python3.7/site-packages/_manylinux.py
RUN python -c 'import sys; sys.path.append(r"/_manylinux.py")'

[stdedos]

Yes; but beware that e.g. py3-cryptography and dependant packages won't like it