Setuptools: Setuptools 42.0.x release multiple breakages

[robvdl]

I'm running into two issues with the setuptools 42.0.x releases, the last known working version for me is 41.6.0

First issue seems to be when it's installing, I am running into an issue when building a wheel:

  Downloading https://files.pythonhosted.org/packages/f7/99/b3a2c6393563ccbe081ffcceb359ec27a6227792c5169604c1bd8128031a/cairocffi-1.1.0.tar.gz (68kB)
    ERROR: Command errored out with exit status 1:
     command: /virtualenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j52ica4p/cairocffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j52ica4p/cairocffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-j52ica4p/cairocffi/pip-egg-info
         cwd: /tmp/pip-install-j52ica4p/cairocffi/
    Complete output (46 lines):
    WARNING: The wheel package is not available.
    WARNING: The wheel package is not available.
    WARNING: The wheel package is not available.
      ERROR: Command errored out with exit status 1:
       command: /virtualenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-4dswyws4/pycparser/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-4dswyws4/pycparser/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-utfcillp
           cwd: /tmp/pip-wheel-4dswyws4/pycparser/
      Complete output (6 lines):
      usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: setup.py --help [cmd1 cmd2 ...]
         or: setup.py --help-commands
         or: setup.py cmd --help

      error: invalid command 'bdist_wheel'
      ----------------------------------------
      ERROR: Failed building wheel for pycparser
    ERROR: Failed to build one or more wheels
    Traceback (most recent call last):
      File "/virtualenv/lib/python3.6/site-packages/setuptools/installer.py", line 119, in fetch_build_egg
        subprocess.check_call(cmd)
      File "/usr/local/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/virtualenv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpfxa_txap', '--quiet', 'pycparser']' returned non-zero exit status 1.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-j52ica4p/cairocffi/setup.py", line 13, in <module>
        'cairocffi/ffi_build.py:ffi_pixbuf']
      File "/virtualenv/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/virtualenv/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/virtualenv/lib/python3.6/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs
        replace_conflicting=True,
      File "/virtualenv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/virtualenv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/virtualenv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/virtualenv/lib/python3.6/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/virtualenv/lib/python3.6/site-packages/setuptools/installer.py", line 121, in fetch_build_egg
        raise DistutilsError(str(e))
    distutils.errors.DistutilsError: Command '['/virtualenv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpfxa_txap', '--quiet', 'pycparser']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Second issue is that it seems to have trouble parsing "graft" in MANIFEST.in files and always expects the folder to be there:

web_1                 |     ERROR: Complete output from command python setup.py egg_info:
web_1                 |     ERROR: running egg_info
web_1                 |     writing top-level names to appname.egg-info/top_level.txt
web_1                 |     writing dependency_links to appname.egg-info/dependency_links.txt
web_1                 |     writing appname.egg-info/PKG-INFO
web_1                 |     writing requirements to appname.egg-info/requires.txt
web_1                 |     warning: manifest_maker: standard file '-c' not found
web_1                 |     
web_1                 |     reading manifest file 'appname.egg-info/SOURCES.txt'
web_1                 |     reading manifest template 'MANIFEST.in'
web_1                 |     error: [Errno 2] No such file or directory: 'appname/*/templates'

Why is it trying to find out if a directory with a wildcard exists? This never used to break until the 42.x releases.

[pganssle]

@robvdl The first issue at least is likely related to the deprecation of easy_install. If you have setup_requires or tests_require dependencies they were being installed by easy_install, which is deprecated. We've made it silently fall back to pip in most situations, which will try to install your dependencies by building a wheel and installing it. If your dependencies don't ship a wheel, don't have a pyproject.toml and you don't have the wheel package installed, it will fail. You can work around this by installing the wheel package, but the preferred solution is for you and your dependencies to add a pyproject.toml that specifies that the build system depends on wheel, and for your dependencies to ship wheels.

I don't know what the other issue is, can you provide a minimal reproducing case?

[robvdl]

I can but give me a few days to get that together, I've got a lot going on.

[AlexBlandin]

Regarding the first issue: I see this on Windows, and ~pip install wheel resolves it (I first reported regarding https://github.com/Kozea/cairocffi/issues/140).

Adding wheel as an dependency, at least as an installation step in the docs, is probably the path of least resistance. Since it's basically a necessary extension, especially with this problem, I can't see why it shouldn't be part of the default setup.

[robvdl]

Yes it seems both issues go away if I install wheel as well.

Agree what Alex is saying that it would be nice to be either added as a dependency or to the docs.

Thanks. Should this issue be closed or remain open?

[pganssle]

@AlexBlandin @robvdl Taking on new dependencies in setuptools is complicated because setuptools is generally at the root of the build tree for everything else, and we're trying to avoid cyclic dependencies. There have been discussions to move the wheel project into setuptools itself rather than having it be a separate project.

Regardless of this, the issue would "fix itself" if we had sufficiently widespread adoption of PEP 517, and were sufficiently confident in PEP 517 that pip could make PEP 517 builds the default, since the default dependencies for all builds are setuptools and wheel.

[AlexBlandin]

It's kinda funny because if you install pip with the get-pip script it will get setuptools _and_ wheel by default, whereas Python doesn't include them by default.

[onlinejudge95]

+1 for the issue, i was unable to install flask-mongoengine==0.9.5 due to the same changes.