Pip: When wheel ABI tag doesn't require debugging, but debugging is enabled, wheel is not instaleld

[wvxvw]

Consider this wheel: https://files.pythonhosted.org/packages/c2/f9/28787754923612ca9bfdffc588daa05580ed70698add063a5629d1a4209d/protobuf-3.6.1-cp36-cp36m-manylinux1_x86_64.whl (I'm not the author, just tried to install it).

It wants ABI version cp36m. However, these are the versions reported as supported on my installation of Python:

[('cp36', 'cp36dm', 'manylinux1_x86_64'),
('cp36', 'cp36dm', 'linux_x86_64'),
('cp36', 'abi3', 'manylinux1_x86_64'),
('cp36', 'abi3', 'linux_x86_64'),
('cp36', 'none', 'manylinux1_x86_64'),
('cp36', 'none', 'linux_x86_64'),
('cp35', 'abi3', 'manylinux1_x86_64'),
('cp35', 'abi3', 'linux_x86_64'),
('cp34', 'abi3', 'manylinux1_x86_64'),
('cp34', 'abi3', 'linux_x86_64'),
('cp33', 'abi3', 'manylinux1_x86_64'),
('cp33', 'abi3', 'linux_x86_64'),
('cp32', 'abi3', 'manylinux1_x86_64'),
('cp32', 'abi3', 'linux_x86_64'),
('py3', 'none', 'manylinux1_x86_64'),
('py3', 'none', 'linux_x86_64'),
('cp36', 'none', 'any'),
('cp3', 'none', 'any'),
('py36', 'none', 'any'),
('py3', 'none', 'any'),
('py35', 'none', 'any'),
('py34', 'none', 'any'),
('py33', 'none', 'any'),
('py32', 'none', 'any'),
('py31', 'none', 'any'),
('py30', 'none', 'any')]

cp36dm should include cp36m, i.e. there is no problem running whatever was compiled for cp36m if host ABI version is cp36dm, but the installation fails. (We are talking about Wheel.supported() from wheel.py).

[chrahunt]

Release libraries being compatible with debug builds is being introduced in Python 3.8 as mentioned in bpo-36465, so I don't think it is necessarily true for Python 3.6 and we probably would not want to make that assumption unless it is guaranteed.

If a current package is compatible with both debug and release, then they could mark wheels with multiple .-separated tags, like cp37m.cp37dm and it should be picked up correctly. For 3.8 I do not know if the default tooling for creating wheels will generate tags like this or if it should be on us the client to ensure the set of allowed tags includes py38m when operating on a debug version.

In the future we will probably also be relying on packaging.tags instead of our built-in version, so it's a discussion that might be better had on the packaging project.

[pradyunsg]

Also, what version of pip are you using?

[wvxvw]

I'm not working on this anymore. This was a year ago. So, I have no idea what version of pip it was. I believe that at the time, I just edited the metadata inside the Wheel to make it appear as if it was a release version to make it work.

we probably would not want to make that assumption unless it is guaranteed.

Why? What is the point? Whoever is running a debug version of Python had compiled it themselves, they do it to test how things work with it. Why make their life more difficult? I mean, if it appears that the API is not compatible between release and debug, so be it, let it crash where it would, if it was not compatible, why try to prevent it?

[pradyunsg]

Let it crash where it would, if it was not compatible, why try to prevent it?

Off the top of my head, I'm guessing that someone will come forward when we do this and tell us that pip shouldn't be crashing and shouldn't allow installation on a different ABI. That argument is also equivalently valid. It's a choice between tradeoffs and someone has to make a call of which tradeoffs are better.

In line with this, if changing this behavior today isn't going to help anyone in a manner that we know of, there's no point in investing effort into fixing this.

[chrahunt]

Personally, I would rather take on the support burden of answering questions like this one than be more flexible in interpreting wheel specs and creating more work for package developers. It is relatively straightforward to answer the question about why a package didn't match compared to debugging a crash, especially since the crash will depend on how the library is used and may be inconsistent.

[pradyunsg]

Given that upstream Python has resolved the issue of mismatched ABI, the failures if we allow such installations would fail in a library-specific manner, OP isn't affected by this anymore (since they no longer work on this) and no other user chimed in or interacted with this issue in the ~11 months this has been open, I'll say maintaining status quo is fine.

Thanks for filing this issue and your patience @wvxvw!

[pradyunsg]

Closing since there's nothing actionable on this issue now.