Setuptools: Setuptools install fails with PEP508 URLs

[davidwilby]

setuptools v 45.1.0
pip v 20.0.2

Running python setup.py install for a setup.py which includes PEP508 format URLs in the install_requires field results in failure.
for example if keras @ https://github.com/MarcBS/keras/archive/master.zip is in the install_requires field then setuptools fails to find it, and looks on PyPI instead, installing a matching package (in this case 'keras').
However, running pip install . for the same setup.py succeeds, similarly with pip install keras @ https://github.com/MarcBS/keras/archive/master.zip.

If setuptools always uses pip to install dependencies, then why does it fail for PEP508 URL dependencies when pip is able to do this?

In summary:

• pip install . works :heavy_check_mark:
• pip install 'keras @ https://github.com/MarcBS/keras/archive/master.zip' works :heavy_check_mark:
• python setup.py install fails :x:

(further to this, PyPI throws an error when uploading a package with a setup.py containing PEP 508 dependency links)

[iambibhas]

Been breaking my head over this for few days. Now that dependency_links support is gone, can no longer add a requirement inside setup.py that depends on a github repo.

[semiversus]

I had the same problem and was confused, that sometimes you find references that PEP508 urls solves the problem - and it does (at least I think so). I hope the following solution is a help for you. Otherwise, I may support you as I was also digging into the problem for days.

What's important: You need an up to date version of pip and setuptools. And here is the catch: When creating a virtual environment, an old version of pip (9.0.1) and setuptools (39.0.1) is used (also when your system pip is newer). Try to update pip and setuptools after generating your virtual environment:

pip install pip setuptools -U

[semiversus]

Hold on - I'm wrong. I still have the same problem as @davidwilby reported. Was using pip install . which is working.

What's interesting is that installing a package containing a PEP508 referenced URL is working (this is involving setup.py).

This example is working and setup.py contains a PEP508 urls:

pip install git+https://github.com/davidwilby/nmt-keras.git

[pganssle]

Our policy to date has been that if using pip install fixes your problem, you should use pip install and we won't fix the issue.

Direct invocations of setup.py install are not the right thing to do, despite the fact that there is a lot of misleading documentation out there indicating that it is.

We to need to work out a plan for actively deprecating and then removing setup.py install to try to get the word out, though.

If someone wants to try to figure out why this is happening, feel free, but it's not clear that we would be willing to accept a PR fixing it if it comes with any additional maintenance burden.

[semiversus]

@pganssle Thanks for the clarification!

[drj11]

hmm, that is an interesting position, given that for many of us python setup.py install is muscle memory and our fingers type it without thinking.

If pip install is the only supported route now, can that policy be documented somewhere?

Does that also apply to pip wheel versus python setup.py bdist wheel too?