Setuptools: 36.2.1 appears to ignore install_requires with extras

Created on 23 Jul 2017 · 12Comments · Source: pypa/setuptools

Travis, Appveyor, and local builds of magic-wormhole started failing about an hour ago, when 36.2.1 was released. Our setup.py has:

      install_requires=[
          "spake2==0.7", "pynacl",
          "six",
          "twisted[tls] >= 17.5.0", # 17.5.0 adds failAfterFailures=
          "autobahn[twisted] >= 0.14.1",
          "automat",
          "hkdf",
          "tqdm >= 4.13.0", # 4.13.0 fixes crash on NetBSD
          "click",
          "humanize",
          "ipaddress",
          "txtorcon >= 0.19.3",
      ],

When I run pip install -e . in a virtualenv with setuptools-36.2.0, I see Autobahn-17.7.1 installed as expected. If I have setuptools-36.2.1, then Autobahn does not get installed. (Twisted does get installed, but it's a dependency of txtorcon).

Source

warner

👍3

Most helpful comment

setup(install_requires=["autobahn[twisted] >= 0.14.1"])
is equivalent to
setup(extras_require={'twisted': ["autobahn >= 0.14.1"]})

I'm pretty sure this statement is wrong.

The former syntax says "this package depends on autobahn with its twisted extra and autobahn should be version 0.14.1 or later". The latter syntax says "this package offers an optional feature called "twisted" and if you ask for thispackage[twisted], this package will additionally depend on autobahn 0.14.1 or later."

jaraco on 24 Jul 2017

👍5

All 12 comments

Should that be pip install -e '.[twisted]'?

benoit-pierre on 23 Jul 2017

Because of the extra specifiers:

setup(install_requires=["autobahn[twisted] >= 0.14.1"])

is equivalent to

setup(extras_require={'twisted': ["autobahn >= 0.14.1"]})

Previously those were ignored.

benoit-pierre on 23 Jul 2017

👎1

Nope, in this case, the un-extra-ed magic-wormhole installation requires both the base autobahn and autobahn's twisted support. (magic-wormhole depends upon twisted directly, but when you ask autobahn to give you its twisted support, it could conceivably depend upon something beyond just twisted, maybe some tx*foo package).

Depending upon autobahn instead of autobahn[twisted] gets you a subset of the functionality that we need.

The only extras that magic-wormhole supports are pip install magic-wormhole[dev], and a :sys_platform=="win32" that gets us some extra windows support.

warner on 23 Jul 2017

Oh, what? Have I been mis-expressing this all this time?

What's the syntax to say "I want dependency X and also X's extra Y"?

warner on 23 Jul 2017

No, I think I'm the one being wrong, and I have introduced a regression...

benoit-pierre on 23 Jul 2017

https://www.python.org/dev/peps/pep-0508/#extras

benoit-pierre on 23 Jul 2017

Whew, ok, good :). I looked at https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies , and it seems to match the way magic-wormhole's setup.py is doing things.

warner on 23 Jul 2017

I'm having the same problem.

$ cat setup.py
from setuptools import setup
setup(
    name="testtest",
    install_requires=['requests[security]']
)

$ pip install .
Processing /Users/bright/Playground/testtest
Installing collected packages: testtest
  Running setup.py install for testtest ... done
Successfully installed testtest-0.0.0

$ pip freeze
testtest==0.0.0

Expected result:

$ pip freeze
asn1crypto==0.22.0
certifi==2017.4.17
cffi==1.10.0
chardet==3.0.4
cryptography==2.0
enum34==1.1.6
idna==2.5
ipaddress==1.0.18
pycparser==2.18
pyOpenSSL==17.2.0
requests==2.18.1
six==1.10.0
testtest==0.0.0
urllib3==1.21.1