Setuptools: No module named 'six'

[davidism]

Flask's Tox/Travis builds all started failing. Looks like a new version of setuptools was just released that has a problem with six.

~pytb
Obtaining file:///home/david/Projects/flask
Installing collected packages: Flask
Running setup.py develop for Flask
Complete output from command /home/david/Projects/flask/.tox/py/bin/python3 -c "import setuptools, tokenize;__file__='/home/david/Projects/flask/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps:
/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/dist.py:336: UserWarning: Normalizing '0.13-dev' to '0.13.dev0'
normalized_version,
Traceback (most recent call last):
File "", line 1, in
File "/home/david/Projects/flask/setup.py", line 109, in
'''
File "/usr/lib64/python3.6/distutils/core.py", line 134, in setup
ok = dist.parse_command_line()
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/dist.py", line 363, in parse_command_line
result = _Distribution.parse_command_line(self)
File "/usr/lib64/python3.6/distutils/dist.py", line 472, in parse_command_line
args = self._parse_command_opts(parser, args)
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/dist.py", line 674, in _parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "/usr/lib64/python3.6/distutils/dist.py", line 528, in _parse_command_opts
cmd_class = self.get_command_class(command)
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/dist.py", line 495, in get_command_class
self.cmdclass[command] = cmdclass = ep.load()
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2303, in load
return self.resolve()
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2309, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/command/develop.py", line 11, in
from setuptools.command.easy_install import easy_install
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 49, in
from setuptools.py27compat import rmtree_safe
File "/home/david/Projects/flask/.tox/py/lib/python3.6/site-packages/setuptools/py27compat.py", line 7, in
import six
ModuleNotFoundError: No module named 'six'
~

Example failed build log: https://travis-ci.org/pallets/flask/jobs/238166427#L242

[thanatos]

I am seeing a very similar traceback; these seem to correspond w/ the release of setuptools==36; I'm trying to repro locally to see if rolling that back fixes.

[davidism]

This doesn't happen if I manually create a virtualenv. In that case, setuptools==36.0.0 is installed but installing packages works fine.

[dstufft]

It looks like an import got missed for being rewritten to import from the vendored packages.

[dmsimard]

Thanks for identifying the issue quickly.

This is currently plaguing most of our python-based jobs, would appreciate a release ASAP once the pull request https://github.com/pypa/setuptools/pull/1043 lands.

[elliterate]

I was able to temporarily work around this, first, by upgrading my CI environment to just short of v36 with the following pre-install command:

pip install --upgrade "setuptools < 36"

Then, by ensuring that my soon-to-be-created virtualenv inherited that version with the following environment variable:

VIRTUALENV_NO_DOWNLOAD=1

[cumber]

I was able to work around this by manually running pip install six in my new virtualenv, before installing anything else. It was able to reuse a cached wheel for six though; not certain that it would have worked otherwise.

[badone]

A quick fix would be nice. This is affecting our builds as well.

[silvana-i]

Adding one more report for this issue. We are using virtualenv, which installs the faulty setuptools version. Afterwards when installing our project dependencies with pip install -r requirements.txt it is choking on the download of a custom package from a locally hosted pypi server. It fails during package collection, so the only thing that we found to fix it is to call pip install six before installing the requirements.txt.

[asottile]

For what it's worth, I made a tox plugin to avoid virtualenv's download behaviour for exactly this reason

Simply pip install tox-virtualenv-no-download alongside tox

[escapewindow]

~virtualenv --no-download PATH will probably work for virtualenv>=14.0.0; that'll prevent hitting pypi for the new setuptools when creating the virtualenv.~
https://virtualenv.pypa.io/en/stable/changes/

Ah, VIRTUALENV_NO_DOWNLOAD is even better, as it doesn't break on virtualenv<14.0.0

[fengxx]

can we have a emergency release for the bugfix?

[fake-name]

It appears you can work around the issue by manually installing six before the dependencies that require it.

[fengxx]

there is a workaround like VIRTUALENV_NO_DOWNLOAD=1 but the issue will cause trouble for virtualenv user who doesn't know the workaround.

[dstufft]

It was pointed out to me that at some point I had been added as a maintainer to setuptools on PyPI, I wasn't aware of that (or had forgotten if I was aware). I've gone ahead and deleted 36.0 from PyPI.

@jaraco I downloaded the files to my computer, if you want to restore 36.0 let me know and I can do it.

[hrw]

@dstufft can you merge pull request with fix and release 36.0.1?

[dstufft]

Jason is the only member of the @pypa/setuptools-developers team, so while I have the technical capability of merging a PR due to being an admin of the PyPA org on Github, I was not explicitly made a committer of the repository and I don't feel comfortable merging that change. Additionally I don't really know the normal release process of setuptools, so I wouldn't feel comfortable cutting a release (and possibly screwing it up again).

Ultimately, the bleeding is stopped and this can wait for @jaraco now.

[graingert]

Smallest passing build #1047

[ilovezfs]

Fix confirmed here as well: https://github.com/Homebrew/homebrew-core/pull/14130.

[pabelanger]

@dstufft If possible, please do not delete packages that have already been tagged and released to pypi. This puts local mirrors in an awkward position of now having an unreleased packaging mirrored.
I know you are not the maintainer but it would have been better to tag 37.0.0 reverting 36.0.0 to 35.x.x

[graingert]

I know you are not the maintainer but it would have been better to tag 37.0.0 reverting 36.0.0 to 35.x.x

that will probably happen soon. Deleting 36 was the fastest way to get travis working again.

[pabelanger]

While it was the fastest way to get TravisCI working again, it breaks other CI systems in the process. It is never possible to delete something from the internet, there will always be a cache somewhere.

[Natim]

@jaraco I downloaded the files to my computer, if you want to restore 36.0 let me know and I can do it.

It won't be possible to upload them again actually...

[andreibacos]

As someone in the process of removing 36 from our internal devpi caches, @pabelanger made a good point.

[Natim]

I don't really know the normal release process of setuptools, so I wouldn't feel comfortable cutting a release

The release process is documented here

I don't think relying on a single person to fix a so central Python package is a good thing. It puts a lot of pressure on their shoulder while we could handle it on our side.

I wouldn't consider it to be a bleeding, its part of the life of a project to ship broken releases once in a while.

The release process of a python project is to release a new version to fix a release.

I would be glad to cut a release myself if you give me necessary permissions.

[matthew-brett]

@Natim - see #1045 for a request to add release candidate builds to the release process.

[Natim]

Yes that's probably a good idea too. Thanks @matthew-brett

[jaraco]

Please see this comment regarding my apology and intention for the project maintenance.