Setuptools: Version 42.0.0 not working with Python 2.7

[lorilew]

Hi, we're getting this error when we use the latest build of setuptools for versions 42.0.0 and 42.0.1.

The command we are executing is:

python setup.py sdist

We fixed our build by pinning version to 41.6.0. We're using Python 2.7 on Ubuntu.

Error message:

FAILED! => {"changed": false, "cmd": ["/opt/susenv/bin/pip2", "install", "--upgrade", "nts-0+0.abcdef1.tar.gz"], "msg": "stdout: Processing ./nts-0+0.abcdef1.tar.gz
Collecting pyspark==2.3.0
  Downloading https://files.pythonhosted.org/packages/58/49/45370cc153a6adcf2c304a3c06e801ed3c9650d0f852e7fde04bd8ffb534/pyspark-2.3.0.tar.gz (211.9MB)

:stderr: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
    ERROR: Command errored out with exit status 1:
     command: /opt/susenv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '\"'\"'/tmp/pip-install-PASxaL/pyspark/setup.py'\"'\"'; __file__='\"'\"'/tmp/pip-install-PASxaL/pyspark/setup.py'\"'\"';f=getattr(tokenize, '\"'\"'open'\"'\"', open)(__file__);code=f.read().replace('\"'\"'\\r\
'\"'\"', '\"'\"'\
'\"'\"');f.close();exec(compile(code, __file__, '\"'\"'exec'\"'\"'))' egg_info --egg-base /tmp/pip-install-PASxaL/pyspark/pip-egg-info
         cwd: /tmp/pip-install-PASxaL/pyspark/
    Complete output (51 lines):
    Could not import pypandoc - required to package PySpark
    WARNING: The wheel package is not available.
    DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
      ERROR: Command errored out with exit status 1:
       command: /opt/susenv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '\"'\"'/tmp/pip-wheel-bXGLeK/pypandoc/setup.py'\"'\"'; __file__='\"'\"'/tmp/pip-wheel-bXGLeK/pypandoc/setup.py'\"'\"';f=getattr(tokenize, '\"'\"'open'\"'\"', open)(__file__);code=f.read().replace('\"'\"'\\r\
'\"'\"', '\"'\"'\
'\"'\"');f.close();exec(compile(code, __file__, '\"'\"'exec'\"'\"'))' bdist_wheel -d /tmp/pip-wheel-7kV1zL
           cwd: /tmp/pip-wheel-bXGLeK/pypandoc/
      Complete output (20 lines):
      Maybe try:

          sudo apt-get install pandoc
      See http://johnmacfarlane.net/pandoc/installing.html
      for installation options
      ---------------------------------------------------------------



      !!! pandoc not found, long_description is bad, don't upload this to PyPI !!!


      no pandoc found, building platform unspecific wheel...
      use 'python setup.py download_pandoc' to download pandoc.
      usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: setup.py --help [cmd1 cmd2 ...]
         or: setup.py --help-commands
         or: setup.py cmd --help

      error: invalid command 'bdist_wheel'
      ----------------------------------------
      ERROR: Failed building wheel for pypandoc
    ERROR: Failed to build one or more wheels
    Traceback (most recent call last):
      File \"<string>\", line 1, in <module>
      File \"/tmp/pip-install-PASxaL/pyspark/setup.py\", line 223, in <module>
        'Programming Language :: Python :: Implementation :: PyPy']
      File \"/opt/susenv/lib/python2.7/site-packages/setuptools/__init__.py\", line 144, in setup
        _install_setup_requires(attrs)
      File \"/opt/susenv/lib/python2.7/site-packages/setuptools/__init__.py\", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File \"/opt/susenv/lib/python2.7/site-packages/setuptools/dist.py\", line 721, in fetch_build_eggs
        replace_conflicting=True,
      File \"/opt/susenv/lib/python2.7/site-packages/pkg_resources/__init__.py\", line 782, in resolve
        replace_conflicting=replace_conflicting
      File \"/opt/susenv/lib/python2.7/site-packages/pkg_resources/__init__.py\", line 1065, in best_match
        return self.obtain(req, installer)
      File \"/opt/susenv/lib/python2.7/site-packages/pkg_resources/__init__.py\", line 1077, in obtain
        return installer(requirement)
      File \"/opt/susenv/lib/python2.7/site-packages/setuptools/dist.py\", line 777, in fetch_build_egg
        return fetch_build_egg(self, req)
      File \"/opt/susenv/lib/python2.7/site-packages/setuptools/installer.py\", line 121, in fetch_build_egg
        raise DistutilsError(str(e))
    distutils.errors.DistutilsError: Command '['/opt/susenv/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpHnv8Lg', '--quiet', 'pypandoc']' returned non-zero exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
"}

[benoit-pierre]

WARNING: The wheel package is not available.

pip is now used (when available) for fetching/building wheels for missing setup_requires/tests_require requirements. Depending on the requirement, you may need to install wheel too (if no compatible wheels are available on PyPI / through dependency links, or PEP 517/518 is not used).

So can you try again with the wheel package installed (or install your build dependencies before running setup.py)?

[pganssle]

@lorilew One thing you may want to consider is adding a pyproject.toml file and using pep517.build to build your sdist. That will ensure that your build dependencies are satisfied before your setup.py is executed.

You do not include your setup.py file, but it seems like what's happening here is that you have setup_requires=["pypandoc"]; a minimal setup.py that demonstrates this issue:

from setuptools import setup
setup(name="mypackage", version="0.0.1", setup_requires=["pypandoc"])

If I create a virtualenv without wheel installed, python setup.py sdist will indeed fail, however, if I add this pyproject.toml file:

[build-system]
requires = ["setuptools>=41.0", "wheel", "pypandoc"]
build-backend = "setuptools.build_meta"

Then python -m pep517.build --source . will correctly build the source distribution.

I think we can probably close this as not a bug in setuptools, but rather an issue with the build environment (and the fact that pypandoc does not ship a wheel for some reason).

I will note that I believe this is only coming to light now because pip always builds a wheel as part of the installation and presumably easy_install does not.

[lorilew]

Thanks for the suggestions, installing wheels has worked and we will look into adding the pyproject.toml file.

[clach04]

Thanks @lorilew - just hit this myself and I couldn't see from the docs for setuptools which version I needed. This was just the information I needed.

For anyone else finding this via google. Ubuntu 16.04 LTS with python 2.7 and pip install setuptools will fail due to needing python3.

pip install setuptools==41.6.0

Resolved this for my project.