Setuptools: AttributeError: Distribution instance has no attribute 'install_requires'

[odyssey4me]

When downgrading setuptools 36.2.0 to 36.1.1 I get the above error. Here's a log output:

2017-07-14 06:03:13.445605 | Installing collected packages: setuptools, pyldap, ldappool, pytz, Babel, MarkupSafe, Jinja2, olefile, Pillow, PyYAML, Pygments, alabaster, six, docutils, sphinx-rtd-theme, snowballstemmer, Sphinx, pbr, bashate, bottle, pycparser, cffi, chardet, coverage, ipaddress, pyasn1, enum34, idna, cryptography, restructuredtext-lint, stevedore, doc8, dulwich, funcsigs, mock, openstackdocstheme, requests, oslosphinx, paramiko, pycrypto, reno, sphinxmark, virtualenv, pyflakes, mccabe, pep8, flake8, hacking, ansible, ansible-lint
2017-07-14 06:03:13.445996 |   Found existing installation: setuptools 36.2.0
2017-07-14 06:03:13.486541 |     Uninstalling setuptools-36.2.0:
2017-07-14 06:03:13.504098 |       Successfully uninstalled setuptools-36.2.0
2017-07-14 06:03:13.510070 |   Rolling back uninstall of setuptools
2017-07-14 06:03:13.702180 | Exception:
2017-07-14 06:03:13.702251 | Traceback (most recent call last):
2017-07-14 06:03:13.702314 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
2017-07-14 06:03:13.702339 |     status = self.run(options, args)
2017-07-14 06:03:13.702398 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/commands/install.py", line 342, in run
2017-07-14 06:03:13.702421 |     prefix=options.prefix_path,
2017-07-14 06:03:13.702480 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/req/req_set.py", line 784, in install
2017-07-14 06:03:13.702496 |     **kwargs
2017-07-14 06:03:13.702556 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/req/req_install.py", line 851, in install
2017-07-14 06:03:13.702588 |     self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
2017-07-14 06:03:13.702650 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
2017-07-14 06:03:13.702672 |     isolated=self.isolated,
2017-07-14 06:03:13.702732 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/wheel.py", line 247, in move_wheel_files
2017-07-14 06:03:13.702750 |     prefix=prefix,
2017-07-14 06:03:13.702810 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/pip/locations.py", line 140, in distutils_scheme
2017-07-14 06:03:13.702833 |     d = Distribution(dist_args)
2017-07-14 06:03:13.702892 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/setuptools/dist.py", line 365, in __init__
2017-07-14 06:03:13.702913 |     self._finalize_requires()
2017-07-14 06:03:13.702974 |   File "/home/jenkins/workspace/gate-openstack-ansible-linters-ubuntu-xenial/.tox/linters/local/lib/python2.7/site-packages/setuptools/dist.py", line 372, in _finalize_requires
2017-07-14 06:03:13.702996 |     if not self.install_requires:
2017-07-14 06:03:13.703029 | AttributeError: Distribution instance has no attribute 'install_requires'

[benoit-pierre]

Yep, sorry... See https://github.com/pypa/setuptools/pull/1085.

[benoit-pierre]

Obviously, it would be great if there was a test to prevent this kind of error in the future. So I've looked into adding one using pytest-virtualenv. But one of the dependencies that end up getting pulled is mock and this result in some tests failing:

___________________________ ERROR at setup of test_ordering ___________________________$
file setuptools/pkg_resources/tests/test_markers.py, line 6
  @mock.patch('platform.python_version', return_value='2.7.10')
  def test_ordering(python_version_mock):
E       fixture 'python_version_mock' not found
>       available fixtures: cache, capfd, capsys, doctest_namespace, monkeypatch, pytest
config, record_xml_property, recwarn, tmpdir, tmpdir_cwd, tmpdir_factory, user_override,
 virtualenv, workspace
>       use 'pytest --fixtures [testpath]' for help on them.
setuptools/pkg_resources/tests/test_markers.py:6
_________________ ERROR at setup of TestBuildCLib.test_build_libraries _________________
file setuptools/setuptools/tests/test_build_clib.py, line 12
      @mock.patch(
              'setuptools.command.build_clib.newer_pairwise_group'
              )
      def test_build_libraries(self, mock_newer):
E       fixture 'mock_newer' not found
>       available fixtures: cache, capfd, capsys, doctest_namespace, monkeypatch, pytes$
config, record_xml_property, recwarn, tmpdir, tmpdir_cwd, tmpdir_factory, user_override$
 virtualenv, workspace
>       use 'pytest --fixtures [testpath]' for help on them.
setuptools/setuptools/tests/test_build_clib.py:12

From the log, I can see that switching to backports.unittest_mock was done for Python 3.2 and 2.6 compatibility but the tests run fine when switching back to mock.

See branch for the changes, note that this include porting the recently added clean install test too (tested locally by reverting 1cebea7e885c357d8f64a021bec4316b7993c50c).

[benoit-pierre]

Oups, I meant to comment on the related PR, sorry!

[odyssey4me]

Odd, I can't seem to isolate when the downgrade works and when it doesn't.

root@setuptools1:~# CURL_CMD="curl --silent --show-error --retry 5"
root@setuptools1:~# OUTPUT_FILE="get-pip.py"
root@setuptools1:~# ${CURL_CMD} https://bootstrap.pypa.io/get-pip.py > ${OUTPUT_FILE} ||\
>     ${CURL_CMD} https://raw.githubusercontent.com/pypa/get-pip/master/get-pip.py > ${OUTPUT_FILE}
root@setuptools1:~# python ${OUTPUT_FILE} pip setuptools wheel
Collecting pip
  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 861kB/s
Collecting setuptools
  Downloading setuptools-36.2.0-py2.py3-none-any.whl (477kB)
    100% |████████████████████████████████| 481kB 2.5MB/s
Collecting wheel
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 7.2MB/s
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-9.0.1 setuptools-36.2.0 wheel-0.29.0
root@setuptools1:~# pip install virtualenv
Collecting virtualenv
  Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
    100% |████████████████████████████████| 1.8MB 585kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-15.1.0
root@setuptools1:~# virtualenv test1
New python executable in /root/test1/bin/python
Installing setuptools, pip, wheel...done.
root@setuptools1:~# source test1/bin/activate
(test1) root@setuptools1:~# pip install setuptools==33.1.1
Collecting setuptools==33.1.1
  Downloading setuptools-33.1.1-py2.py3-none-any.whl (472kB)
    100% |████████████████████████████████| 481kB 2.2MB/s
Installing collected packages: setuptools
  Found existing installation: setuptools 36.2.0
    Uninstalling setuptools-36.2.0:
      Successfully uninstalled setuptools-36.2.0
Successfully installed setuptools-33.1.1
(test1) root@setuptools1:~# deactivate
root@setuptools1:~# pip install setuptools==33.1.1
Collecting setuptools==33.1.1
  Using cached setuptools-33.1.1-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 36.2.0
    Uninstalling setuptools-36.2.0:
      Successfully uninstalled setuptools-36.2.0
Successfully installed setuptools-33.1.1

[benoit-pierre]

It only happens when downgrading/upgrading from 36.2.0 and from sources.

[odyssey4me]

OK, I've confirmed that if doing both an install of something from a git source and downgrading setuptools at the same time causes the failure.

[lukeyeager]

Really looking forward to getting our builds running again (e.g. https://travis-ci.org/NVIDIA/DIGITS/jobs/254616137). Since setuptools is such a critical piece of the Python infrastructure, I assume I can wait a day or two for a critical fix to be pushed out, rather than hacking our infrastructure to work around the issue for the long term.

[mimischi]

Is this issue supposed to be fixed? Still experiencing the error when building on Travis CI. Or is there something wrong on their side?

[jaraco]

Yes, it should be fixed. It's possible you've managed to somehow pick up the issue from an old build or you have the same error but with a different cause. Probably better to file a new issue unless you can ascertain that your symptoms have the same cause as those identified and fixed herein.

[marius92mc]

I still experience the same issue as @mimischi described.

[benoit-pierre]

@mimischi, @marius92mc: there's no way to know if you're experiencing the same issue with the limited information you provide. As noted above, the problem should only occur when a package is built from source and setuptools is also downgraded/upgraded from 36.2.0 as part of the same pip install invocation. The workaround is to downgrade/update setuptools by itself from wheel.

[JasonMWhite]

For what it's worth, I also just encountered the same issue on Travis-CI. I had specified setuptools==36.5.0 in my requirements.txt file. Travis-CI was uninstalling 36.2.0 and errored out when doing so.
https://travis-ci.org/GoC-Spending/fuzzy-tribble/builds/277605046

  Found existing installation: setuptools 36.2.0
    Uninstalling setuptools-36.2.0:
      Successfully uninstalled setuptools-36.2.0
  Rolling back uninstall of setuptools
Exception:
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/pip/locations.py", line 140, in distutils_scheme
    d = Distribution(dist_args)
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/setuptools/dist.py", line 365, in __init__
    self._finalize_requires()
  File "/home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages/setuptools/dist.py", line 372, in _finalize_requires
    if not self.install_requires:
AttributeError: 'Distribution' object has no attribute 'install_requires'

[benoit-pierre]

@JasonMWhite: https://github.com/pypa/setuptools/issues/1086#issuecomment-329216463

[JasonMWhite]

This checks out. A minimum reproducible test case had a requirements.txt file of just setuptools==36.5.0 and pylint==1.7.2. Looks like pylint is compiling wrapt in the installation, which is sufficient to trigger this bug.

Added a new line to my .travis.yml file to pip install setuptools==36.5.0 _before_ pip install -r requirements.txt solved the problem for me, just in case anyone else hits the same issue.

[ob-ivan]

the problem should only occur when a package is built from source and setuptools is also downgraded/upgraded from 36.2.0 as part of the same pip install invocation.

This is exactly the situation I'm encountering. I'm trying to install https://github.com/Nanoseb/ncTelegram on Fedora following its instructions on the readme. When I call:

$ sudo pip3 install --upgrade https://github.com/Nanoseb/ncTelegram/archive/0.9.2.tar.gz

This is what I get:

WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting https://github.com/Nanoseb/ncTelegram/archive/0.9.2.tar.gz
  Downloading https://github.com/Nanoseb/ncTelegram/archive/0.9.2.tar.gz
Collecting pytg==0.4.5 (from ncTelegram==0.9.1)
  Using cached pytg-0.4.5.tar.gz
Collecting DictObject (from pytg==0.4.5->ncTelegram==0.9.1)
  Using cached DictObject-0.1.3.tar.gz
Collecting luckydonald-utils>=0.17 (from pytg==0.4.5->ncTelegram==0.9.1)
  Using cached luckydonald-utils-0.54.tar.gz
Requirement already up-to-date: pip in /usr/lib/python3.6/site-packages (from luckydonald-utils>=0.17->pytg==0.4.5->ncTelegram==0.9.1)
Collecting setuptools (from luckydonald-utils>=0.17->pytg==0.4.5->ncTelegram==0.9.1)
  Using cached setuptools-36.5.0-py2.py3-none-any.whl
Installing collected packages: setuptools, luckydonald-utils, DictObject, pytg, ncTelegram
  Found existing installation: setuptools 36.2.0
    Uninstalling setuptools-36.2.0:
      Successfully uninstalled setuptools-36.2.0
  Rolling back uninstall of setuptools
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3.6/site-packages/pip/commands/install.py", line 365, in run
    strip_file_prefix=options.strip_file_prefix,
  File "/usr/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/lib/python3.6/site-packages/pip/req/req_install.py", line 854, in install
    strip_file_prefix=strip_file_prefix
  File "/usr/lib/python3.6/site-packages/pip/req/req_install.py", line 1069, in move_wheel_files
    strip_file_prefix=strip_file_prefix,
  File "/usr/lib/python3.6/site-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/usr/lib/python3.6/site-packages/pip/locations.py", line 140, in distutils_scheme
    d = Distribution(dist_args)
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 365, in __init__
    self._finalize_requires()
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 372, in _finalize_requires
    if not self.install_requires:
AttributeError: 'Distribution' object has no attribute 'install_requires'

The workaround is to downgrade/update setuptools by itself from wheel.

Sorry, I'm complete zero in python world. What will be the command line to run to achieve that?

UPD. Ok, a quick RTFM gave the answer:

$ sudo pip3 install setuptools -U

Hope this helps anyone in the same situation.