Setuptools: SyntaxError: invalid syntax on Python 3.5.2

[rsur-dev]

I get the following error:

File "xxxxxxxxxxxxxxxxxxxxxxxxxx/ve3/lib/python3.5/site-packages/setuptools/command/install_scripts.py", line 17, in run
import setuptools.command.easy_install as ei
File "xxxxxxxxxxxxxxxxxxxxxxxxxx/ve3/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 719
):
^
SyntaxError: invalid syntax

I think is related to the trailing comma on line 718 on setuptools/command/easy_install.py:

def process_distribution( # noqa: C901
  self, requirement, dist, deps=True, *info,
):

I'm using Python 3.5.2.

Thanks,

[konradwudkowski]

I've got exactly the same problem and indeed the trailing comma on line 718 is the culprit on python 3.5. Would it be possible to remove it?

[philroche]

This appears to have been introduced in 51.3.0 with commit https://github.com/pypa/setuptools/commit/fc891f5cf6d93ad533e2afb5e15a2952408ab358

[skaiaa]

+1 same error, python 3.5

[rvanlaar]

+1 same error, python 3.5

Used for Ubuntu 16.04.

[di]

cc @webknjaz

[webknjaz]

Ah, setuptools dropped support for Python 3.5 about 1.5 months ago: https://setuptools.readthedocs.io/en/latest/history.html#v51-0-0.

The metadata clearly says >= 3.6: https://github.com/pypa/setuptools/blob/8222d6f/setup.cfg#L30.

So how did you all get a version that pip wasn't even supposed to pull-in? Can this be another instance of Ubuntu/Debian packaging essential Python packages wrong? https://gist.github.com/tiran/2dec9e03c6f901814f6d1e8dad09528e

If that's the case, I suggest folks complain to their distro packagers because I doubt that the setuptools upstream has the capacity to deal with this, nor can it influence the correctness of packaging debs.

But if you somehow got it via pip, this could be their bug.

I'd say that setuptools-wise, it's a "won't fix" because it's not a bug: testing has been dropped for Python 3.5 from the CIs. And also it's been EOL for almost 5 months already.

cc @jaraco

[webknjaz]

This appears to have been introduced in 51.3.0

https://pypi.org/project/setuptools/51.3.0/ shows that Requires-Python is >= 3.6 proving that the dists are correct so any tools respecting that metadata should be able to exclude this version from the resolution under Python envs that don't match the requirement.

[jaraco]

Some ways that pip (or other installers) fail to recognize the supported versions:

• Pip < 9 is used.
• An index doesn't yet honor PEP 503.

One way to work around the issue in these environments is to pin to a specific setuptools version.

It looks like 51.3.3 is the latest version that supports Python 3.5:

draft $ python3.6 -m pip-run setuptools -- -c 'import setuptools.command.easy_install'
Collecting setuptools
  Using cached setuptools-51.3.3-py3-none-any.whl (786 kB)
Installing collected packages: setuptools
Successfully installed setuptools-51.3.3

Hope that helps.

[dvolosnykh]

As not an expert in Python infrastructure, but the user affected by the issue I have tried the suggestion by @jaraco with Python 3.5.2 on Ubuntu 16.04 and it did not work for me. See my steps below:

$ docker container run -it --rm ubuntu:16.04 bash
root@adfae42b8751:/# apt update
...
root@adfae42b8751:/# apt install python3
...
root@adfae42b8751:/# python3 --version
Python 3.5.2
root@adfae42b8751:/# python3 -m pip-run setuptools -- -c 'import setuptools.command.easy_install'
/usr/bin/python3.5: No module named pip-run

Finally, the following Dockerfile fits my purpose well:

FROM ubuntu:16.04

RUN apt update && \
  apt install -y python3-pip

ENV PYTHONWARNINGS=ignore:DEPRECATION
RUN pip3 install --upgrade 'pip<21' 'setuptools<51'
RUN pip3 install pyyaml

[ailin-nemui]

we've just run into this issue on github actions, where pip3 install setuptools when done on Ubuntu 16.04 will pull in the unsupported package

[JulienPalard]

we've just run into this issue on github actions, where pip3 install setuptools when done on Ubuntu 16.04 will pull in the unsupported package

Can you try to first python3 -m pip install --upgrade pip?

[ailin-nemui]

in confirmation of the points made by jaraco, Ubuntu is using pip 8.1.1

[philroche]

My issues were also on Ubuntu 16.04 Xenial and a bug has been filed on launchpad @

https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1912248

[philroche]

I have found the following workaround on Ubuntu 16.04 Xenial

Do not install setuptools on virtualenv creation:

virtualenv -p /usr/bin/python3 venv/py3 --verbose --no-setuptools

Install a supported setuptools version after virtualenv creation and activation:

python3 -m pip install --upgrade 'setuptools; python_version >= "3.6"' 'setuptools<51.3.0; python_version < "3.6" and python_version >= "3.0"'

Ignoring setuptools: markers 'python_version >= "3.6"' don't match your environment
Ignoring setuptools: markers 'python_version < "3.0"' don't match your environment
Collecting setuptools<51.3.0
Using cached setuptools-50.3.2-py3-none-any.whl (785 kB)
Installing collected packages: setuptools
Successfully installed setuptools-50.3.2

or by adding the following to your requirements.txt

setuptools; python_version >= "3.6"
setuptools<51.3.0; python_version < "3.6" and python_version >= "3.0"

This results in python3 -m pip list:

pip 20.3.3
setuptools 50.3.2
wheel 0.36.2

[webknjaz]

@philroche or if you have virtualenv v20+, you could also use the seeders mechanism: https://virtualenv.pypa.io/en/stable/user_guide.html#seeders.

[philroche]

@philroche or if you have virtualenv v20+, you could also use the seeders mechanism: https://virtualenv.pypa.io/en/stable/user_guide.html#seeders.

@webknjaz As far as I can tell changing the seeded version of setuptools is not supported in virtualenv version 15.0.1 which ships with Ubuntu 16.04

$ virtualenv --version
15.0.1
$ virtualenv --help
Usage: virtualenv.py [OPTIONS] DEST_DIR

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         Increase verbosity.
  -q, --quiet           Decrease verbosity.
  -p PYTHON_EXE, --python=PYTHON_EXE
                        The Python interpreter to use, e.g.,
                        --python=python2.5 will use the python2.5 interpreter
                        to create the new environment.  The default is the
                        python2 interpreter on your path (e.g.
                        /usr/bin/python2)
  --clear               Clear out the non-root install and start from scratch.
  --no-site-packages    DEPRECATED. Retained only for backward compatibility.
                        Not having access to global site-packages is now the
                        default behavior.
  --system-site-packages
                        Give the virtual environment access to the global
                        site-packages.
  --always-copy         Always copy files rather than symlinking.
  --unzip-setuptools    Unzip Setuptools when installing it.
  --relocatable         Make an EXISTING virtualenv environment relocatable.
                        This fixes up scripts and makes all .pth files
                        relative.
  --no-setuptools       Do not install setuptools in the new virtualenv.
  --no-pip              Do not install pip in the new virtualenv.
  --no-wheel            Do not install wheel in the new virtualenv.
  --extra-search-dir=DIR
                        Directory to look for setuptools/pip distributions in.
                        This option can be used multiple times.
  --download            Download preinstalled packages from PyPI.
  --no-download, --never-download
                        Do not download preinstalled packages from PyPI.
  --prompt=PROMPT       Provides an alternative prompt prefix for this
                        environment.
  --setuptools          DEPRECATED. Retained only for backward compatibility.
                        This option has no effect.
  --distribute          DEPRECATED. Retained only for backward compatibility.
                        This option has no effect.

[webknjaz]

That's why I said it's for 20+

[philroche]

You did. Thank you. Just adding detail for others on Ubuntu 16.04 that have been affected.

[rsur-dev]

Sorry I didn't check the required python before writing the issue.

I also have an old docker image with ubuntu 16.04 that stopped working.
I'll update my image.

Thanks.

[sabaini]

Fwiw another workaround, I'm now using python3-venv and upgrading pip:

ubuntu@juju-1c6975-4:~$ lsb_release -c
Codename:       xenial
ubuntu@juju-1c6975-4:~$ sudo apt install python3-venv
...
ubuntu@juju-1c6975-4:~$ python3 -m venv v
ubuntu@juju-1c6975-4:~$ v/bin/pip install -U pip 
...
ubuntu@juju-1c6975-4:~$ v/bin/pip install pyyaml
Successfully installed pyyaml-5.3.1

[jaraco]

Another recommendation I might suggest - don't install Setuptools at all. Most projects, if installed with pip --use-pep517 don't require setuptools to be installed prior to performing the install with pip. This requires a later version of pip with pep517 support (18 maybe?), but moves away from the implied dependency on setuptools.

[philroche]

Following up on my initial workaround. Here is a one liner

virtualenv -p /usr/bin/python3 venv/py3 --verbose --no-download && . venv/py3/bin/activate && python3 -m pip install --upgrade 'setuptools; python_version >= "3.6"' 'setuptools<51.3.0; python_version < "3.6" and python_version >= "3.0"' pip wheel && python3 -m pip list

Which uses the bundled pip, wheel and setuptools initially but then upgrades pip, wheel and setuptools using environment markers to ensure the supported version is installed.