Pip: SyntaxError in vendored html5lib

Created on 2 Nov 2016  Â·  13Comments  Â·  Source: pypa/pip

If you pip install pip==9 -v, it ends with:

Installing collected packages: pip
  *** Error compiling '/tmp/pip-build-xvt_4sme/pip/pip/_vendor/html5lib/treewalkers/etree.py'...
    File "/tmp/pip-build-xvt_4sme/pip/pip/_vendor/html5lib/treewalkers/etree.py", line 13
      from pip._vendor. import string_types
                             ^
  SyntaxError: invalid syntax


  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip to 755
  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip3 to 755
  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip3.5 to 755
Successfully installed pip-9.0.0
Cleaning up...

cc @dstufft

auto-locked

Most helpful comment

This problem is breaking conda-build (the library which builds conda recipes). So it's more than just noise. I'm not sure exactly why but I suspect it's calling -mcompileall. I'm looking forward to 9.0.1.

All 13 comments

By the way, related to #4055, it might be interesting to commit the scripts used currently for the different vendoring.

Hi,
i also receive similar install 9.0
it seems to install - with these errors

File "build\bdist.win32\egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

File "c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

Processing pip-9.0.0.tar.gz
Writing c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\setup.cfg
Running pip-9.0.0\setup.py -q bdist_egg --dist-dir c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\egg-dist-tmp-i3bjdv
removing 'c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg' (and everything under it)
creating c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Extracting pip-9.0.0-py2.7.egg to c:\program files (x86)\python27\arcgis10.3\lib\site-packages
pip 9.0.0 is already the active version in easy-install.pth
Installing pip-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts

Installed c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Processing dependencies for pip==9.0.0
Finished processing dependencies for pip==9.0.0

Pip still works after this error yes? Or is it completely broken?

Sent from my iPhone

On Nov 2, 2016, at 6:46 PM, rudyvogt [email protected] wrote:

Hi,
i also receive similar install 9.0
it seems to install - with these errors

File "build\bdist.win32\egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

File "c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

Processing pip-9.0.0.tar.gz
Writing c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\setup.cfg
Running pip-9.0.0\setup.py -q bdist_egg --dist-dir c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\egg-dist-tmp-i3bjdv
removing 'c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg' (and everything under it)
creating c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Extracting pip-9.0.0-py2.7.egg to c:\program files (x86)\python27\arcgis10.3\lib\site-packages
pip 9.0.0 is already the active version in easy-install.pth
Installing pip-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts

Installed c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Processing dependencies for pip==9.0.0
Finished processing dependencies for pip==9.0.0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Apparently it globally works, it just makes some noise during the installation.
No urgency here 😊

Ok. I was SUPER confused how none of our tests or manual testing caught this, but if it works it just poops out a message with -v that makes sense.

Sent from my iPhone

On Nov 2, 2016, at 7:11 PM, Xavier Fernandez [email protected] wrote:

Apparently it globally works, it just makes some noise during the installation.
No urgency here 😊

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@dstufft
yes it seems to work after install - looks like a bit of noise. I could install other packages/upgrade
i have python 2.7.8 installed

Pip still works after this error yes? Or is it completely broken?

Calling python2.7 -mcompileall on the virtualenv folder fails with an error because of this.
Perhaps, compiling pip should become a test.

This problem is breaking conda-build (the library which builds conda recipes). So it's more than just noise. I'm not sure exactly why but I suspect it's calling -mcompileall. I'm looking forward to 9.0.1.

Just tested 9.0.1 seems to work 100% - Great work for fixing it so quickly

Seeing this error simply installing pip using easy_install on centos 6.9.

$ docker run -ti centos:6.9 bash
Unable to find image 'centos:6.9' locally
6.9: Pulling from library/centos
993c50d47469: Pull complete 
Digest: sha256:5cf988fbf143af398f879bd626ee677da3f8d229049b7210790928a02613ab26
Status: Downloaded newer image for centos:6.9
[root@b3dbcef03a1b /]# 
[root@b3dbcef03a1b /]# yum install -y python-setuptools && easy_install pip
Loaded plugins: fastestmirror, ovl
Setting up Install Process
base                                                                                                                 | 3.7 kB     00:00     
base/primary_db                                                                                                      | 4.7 MB     00:01     
extras                                                                                                               | 3.4 kB     00:00     
extras/primary_db                                                                                                    |  30 kB     00:00     
updates                                                                                                              | 3.4 kB     00:00     
updates/primary_db                                                                                                   | 8.1 MB     00:03     
Resolving Dependencies
--> Running transaction check
---> Package python-setuptools.noarch 0:0.6.10-4el6_9 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================
 Package                                Arch                        Version                              Repository                    Size
============================================================================================================================================
Installing:
 python-setuptools                      noarch                      0.6.10-4el6_9                        updates                      336 k

Transaction Summary
============================================================================================================================================
Install       1 Package(s)

Total download size: 336 k
Installed size: 1.5 M
Downloading Packages:
python-setuptools-0.6.10-4el6_9.noarch.rpm                                                                           | 336 kB     00:00     
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Importing GPG key 0xC105B9DE:
 Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <[email protected]>
 Package: centos-release-6-9.el6.12.3.x86_64 (@CentOS/6.9)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-setuptools-0.6.10-4el6_9.noarch                                                                                   1/1 
  Verifying  : python-setuptools-0.6.10-4el6_9.noarch                                                                                   1/1 

Installed:
  python-setuptools.noarch 0:0.6.10-4el6_9                                                                                                  

Complete!
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 10.0.1
Downloading https://files.pythonhosted.org/packages/ae/e8/2340d46ecadb1692a1e455f13f75e596d4eab3d11a57446f08259dee8f02/pip-10.0.1.tar.gz#sha256=f2bd08e0cd1b06e10218feaf6fef299f473ba706582eb3bd9d52203fdbd7ee68
Processing pip-10.0.1.tar.gz
Running pip-10.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-WSSKBu/pip-10.0.1/egg-dist-tmp-YWu5LC
/usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching '.landscape.yml'
warning: no previously-included files found matching 'src/pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching '*-requirements.txt'
warning: no previously-included files found matching 'appveyor.yml'
warning: no previously-included files found matching 'src/pip/_vendor/six'
warning: no previously-included files matching '*.pyi' found under directory 'src/pip/_vendor'
no previously-included directories found matching '.github'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'news'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/pep425tags.py', 84, 35, "    if not soabi and impl in {'cp', 'pp'} and hasattr(sys, 'maxunicode'):\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/baseparser.py', 156, 37, '        section_items = {name: [] for name in override_order}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/utils/misc.py', 849, 29, '    reverse = {value: key for key, value in enums.items()}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/resolve.py', 35, 35, '    _allowed_strategies = {"eager", "only-if-needed", "to-satisfy-only"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/index.py', 526, 43, '                        {str(c.version) for c in all_candidates},\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/cache.py', 37, 35, '        _valid_formats = {"source", "binary"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/configuration.py', 101, 27, '            variant: [] for variant in self._override_order\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/wheel.py', 173, 31, '        parent_dir: scripts for parent_dir, scripts in grouped_by_dir.items()\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/basecommand.py', 195, 21, '                } for name in [\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/list.py', 170, 23, '        return {pkg for pkg in packages if pkg.key not in dep_keys}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/__init__.py', 42, 30, 'commands_dict = {c.name: c for c in commands_order}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/freeze.py', 11, 18, "DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel'}\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/download.py', 142, 52, "        binary_only = FormatControl(set(), {':all:'})\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/req/req_uninstall.py', 137, 37, '        os.path.join(folder, "*") for folder in folders\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/req/req_install.py', 98, 51, '                pkg_resources.safe_extra(extra) for extra in req.extras\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/compat.py', 180, 24, 'stdlib_pkgs = {"python", "wsgiref", "argparse"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_vendor/html5lib/_trie/_base.py', 16, 21, '        return {x for x in keys if x.startswith(prefix)}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/pep425tags.py', 84, 35, "    if not soabi and impl in {'cp', 'pp'} and hasattr(sys, 'maxunicode'):\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/baseparser.py', 156, 37, '        section_items = {name: [] for name in override_order}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/resolve.py', 35, 35, '    _allowed_strategies = {"eager", "only-if-needed", "to-satisfy-only"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/index.py', 526, 43, '                        {str(c.version) for c in all_candidates},\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/cache.py', 37, 35, '        _valid_formats = {"source", "binary"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/configuration.py', 101, 27, '            variant: [] for variant in self._override_order\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/wheel.py', 173, 31, '        parent_dir: scripts for parent_dir, scripts in grouped_by_dir.items()\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/basecommand.py', 195, 21, '                } for name in [\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/compat.py', 180, 24, 'stdlib_pkgs = {"python", "wsgiref", "argparse"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/utils/misc.py', 849, 29, '    reverse = {value: key for key, value in enums.items()}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/list.py', 170, 23, '        return {pkg for pkg in packages if pkg.key not in dep_keys}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/__init__.py', 42, 30, 'commands_dict = {c.name: c for c in commands_order}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/freeze.py', 11, 18, "DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel'}\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/download.py', 142, 52, "        binary_only = FormatControl(set(), {':all:'})\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/req/req_uninstall.py', 137, 37, '        os.path.join(folder, "*") for folder in folders\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/req/req_install.py', 98, 51, '                pkg_resources.safe_extra(extra) for extra in req.extras\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_vendor/html5lib/_trie/_base.py', 16, 21, '        return {x for x in keys if x.startswith(prefix)}\n'))

Adding pip 10.0.1 to easy-install.pth file
Installing pip script to /usr/bin
Installing pip2.6 script to /usr/bin
Installing pip2 script to /usr/bin

Installed /usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip

pip 10 dropped support for Python 2.6. You can install pip 9 (current latest 9.0.3) if you want to use it on Python 2.6.

Further, you seem to be coming from a really old version of setuptools.

If you want to install pip on Python 2.6, one way is to use https://bootstrap.pypa.io/2.6/get-pip.py .

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Was this page helpful?
0 / 5 - 0 ratings