Setuptools: Add support for wheels to setuptools easy_install....

[ghost]

Originally reported by: pombredanne (Bitbucket: pombredanne, GitHub: pombredanne)

---

what would it take? This could help enable wheels in buildout too BTW
https://github.com/buildout/buildout/issues/144

What I mean by this is to be able to install wheels like we can install eggs (wheel adds bdist_wheel to setuptools already)

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/78

[ghost]

_Original comment by_ RonnyPfannschmidt (Bitbucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt):

---

multi version installs of wheels seem possible by just treating them as zip-unsafe eggs with a different extension

actually it would probably be best if setuptools just did that

[ghost]

_Original comment by_ jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

I regularly get requests for this feature.

[ghost]

_Original comment by_ pombredanne (Bitbucket: pombredanne, GitHub: pombredanne):

---

The current state of things with pip works for me

[ghost]

_Original comment by_ pombredanne (Bitbucket: pombredanne, GitHub: pombredanne):

---

I am closing this now. pip is the way to go and this is also possibly supported now in the new setuptools anyway.

[ghost]

_Original comment by_ pombredanne (Bitbucket: pombredanne, GitHub: pombredanne):

---

Or could it be that towards a better segregation of duties between the different tools easy_install and pip would merge? :P

[leorochael]

In a recent thread [1] on distutils-sig, this has come up again, but in the context of buildout missing support for wheels.

[1] Huge multi-branching thread, really. See threads about "Deprecating file extensions", "Future of buildout" and "What roles eggs still play^

From what I can see, the following changes are needed:

In setuptools/package_index.py consider wheels for download, the same way .exe files are considered for download.

In setuptools/command/easy_install.py, treat wheel files the same way .exe files are treated, by converting them to unzipped eggs, including mapping wheel_name-1.0.data/platlib and wheel_name-1.0.data/purelib files as needing to be mapped to the root of the wheel.

[jaraco]

I'm not even sure they need to be treated as zip-unsafe eggs. My understanding is that wheel files are meant to be a suitable replacement for eggs, allowing them to be added to sys.path through the same easy-install.pth mechanism and also readily deleted for uninstall.

Regardless, I welcome a PR to address this issue. It would ease the disparity between the two installers.

[leorochael]

In the threads I mentioned above, the "powers that be" were very insistent that wheels are not a full replacement of eggs, just of the installation aspect of eggs.

Many wheels can be added still zipped to the PYTHONPATH and their code will be available to Python (but not all, se below), but I'm not sure if pkg_resources would be able to read their metadata in this format.

However, a careful reading of the wheel spec shows that wheels that contain a directory like wheel_name-1.0.data/platlib or wheel_name-1.0.data/purelib are explicitly incompatible with being added to the PYTHONPATH, zipped or not. They would need to be unpacked and theses subdirectories moved to the root of the unpacking directory before this directory could be added to the PYTHONPATH.

At this point, the resulting directory looks pretty much like an unzipped egg, except for the wheel_name-1.0.dist-info directory instead of EGG-INFO, but it looks like pkg_resources can already handle this format of metadata as well.

Anyway, by following the way .exe files (created by bdist_wininst) are handled by easy_install and package_index, (including the directory mapping done by get_exe_prefixes() to handle purelib and platlib directories) one should be able to come up with a PR that will enable setuptools to handle wheel files that will benefit all users of setuptools.

Unfortunately, for the moment I'm out of round 'tuits to implement this PR myself.

[leorochael]

In the buildout issue mentioned just above, @dholth had this to say about adding wheel installation support to setuptools:

It should be easy to add wheel install support to setuptools or easy_install or whatever buildout may need. pkg_resources was the first tool to support dist-info directories, and once they are unpacked they look exactly like eggs in your pkg_resources.working_set. There's code from the wheel install command that could be called and is fairly setuptools-onic. Use wheel.install.WheelFile() instances which are sortable by how desirable the particular wheel is, this works with only the filename. Grab WheelFile.install_paths(), change them all to point to the single directory in which it should be installed egg-style, pass to WheelFile.install(overrides={...}), then generate console scripts as usual.

I guess this would mean vendoring wheel just, like packaging.

The only part that might need a little love is wheel.pep425tags which generates a slightly different list of supported tags than pip.pep425tags, mainly pip supports more OSX tags. They should probably just be kept identical.

The pep425tags part should probably be treated by packaging somehow, so that both pip and setuptools (and wheel?) treat the compatibility tags the same. Currently in setuptools, the compatibility tags are handled by pkg_resources.Environment.can_add() AFAICT, and for the purposes of supporting wheel, and treating it as just as downloadable as any other distribution, we might not need to change anything, though we might want to defer to an official pep425tags implementation in the future.

As for vendoring wheel, we need to weight the cost of that versus the benefit of using it for just copying some files around (and the cost of duplicating this code into another, potentially different, implementation).

[dholth]

Or you could install wheel when you need to install wheels, just like pip installs wheel when it needs to build them. Wheel is probably already installed.

[leorochael]

Historically, setuptools has been self-sufficient in the formats it knows how to install, this is to allow other systems (e.g. buildout) to easily bootstrap themselves by depending on setuptools only.

This means that, although you can easily add other build formats, like bdist_wheel, by adding an entry point, setuptools currently doesn't have an entry point for installing different binary formats.

Still, if one is feeling adventurous, one could create such entry point configurability for installation formats on setuptools and take the opportunity to spin off the installation of bdist_wininst (and perhaps the building of bdist_wininst as well) into another package that we could put to pasture.

[jimfulton]

FYI, we've made some progress on this on the buildout side.

For now, this uses an extension:

https://github.com/buildout/wheel/blob/master/src/buildout/wheel/__init__.py

That monkey patches setuptools to:

• treat .whl as a source extension and

• augments interpret_distro_name to parse the project/version correctly.

When buildout sees a .whl distro, it converts it to an egg and the installs that.

The egg conversion currently uses humpty, which uses distlib, which seems heavy and buggy. I'd like us to get rid of those dependencies, if practical.

At a minimum, we'll probably want to make a PR to optionally have setuptools treat wheels as source releases, to avoid monkey patching, although ideally we'd provide a PR to provide wheel support proper.

[benoit-pierre]

This is supported starting with 38.2.0.