Pip: SyntaxError in vendored html5lib

[xavfernandez]

If you pip install pip==9 -v, it ends with:

Installing collected packages: pip
  *** Error compiling '/tmp/pip-build-xvt_4sme/pip/pip/_vendor/html5lib/treewalkers/etree.py'...
    File "/tmp/pip-build-xvt_4sme/pip/pip/_vendor/html5lib/treewalkers/etree.py", line 13
      from pip._vendor. import string_types
                             ^
  SyntaxError: invalid syntax


  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip to 755
  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip3 to 755
  changing mode of /home/xfernandez/.virtualenvs/tmp-8096e8a0e248648/bin/pip3.5 to 755
Successfully installed pip-9.0.0
Cleaning up...

cc @dstufft

[hannes-ucsc]

https://github.com/pypa/pip/blob/c8e8a99b7a6f9404536bc9d895a1a42a060f7f91/pip/_vendor/html5lib/treewalkers/etree.py#L13

[xavfernandez]

By the way, related to #4055, it might be interesting to commit the scripts used currently for the different vendoring.

[rudyvogt]

Hi,
i also receive similar install 9.0
it seems to install - with these errors

File "build\bdist.win32\egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

File "c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

Processing pip-9.0.0.tar.gz
Writing c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\setup.cfg
Running pip-9.0.0\setup.py -q bdist_egg --dist-dir c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\egg-dist-tmp-i3bjdv
removing 'c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg' (and everything under it)
creating c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Extracting pip-9.0.0-py2.7.egg to c:\program files (x86)\python27\arcgis10.3\lib\site-packages
pip 9.0.0 is already the active version in easy-install.pth
Installing pip-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts

Installed c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Processing dependencies for pip==9.0.0
Finished processing dependencies for pip==9.0.0

[dstufft]

Pip still works after this error yes? Or is it completely broken?

Sent from my iPhone

On Nov 2, 2016, at 6:46 PM, rudyvogt [email protected] wrote:

Hi,
i also receive similar install 9.0
it seems to install - with these errors

File "build\bdist.win32\egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

File "c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg\pip_vendor\html5lib\treewalkers\etree.py", line 13
from pip._vendor. import string_types
^
SyntaxError: invalid syntax

Processing pip-9.0.0.tar.gz
Writing c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\setup.cfg
Running pip-9.0.0\setup.py -q bdist_egg --dist-dir c:\users\adm-ru~1\appdata\local\temp\easy_install-hnrnav\pip-9.0.0\egg-dist-tmp-i3bjdv
removing 'c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg' (and everything under it)
creating c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Extracting pip-9.0.0-py2.7.egg to c:\program files (x86)\python27\arcgis10.3\lib\site-packages
pip 9.0.0 is already the active version in easy-install.pth
Installing pip-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.7.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2-script.py script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts
Installing pip2.exe.manifest script to C:\Program Files (x86)\Python27\ArcGIS10.3\Scripts

Installed c:\program files (x86)\python27\arcgis10.3\lib\site-packages\pip-9.0.0-py2.7.egg
Processing dependencies for pip==9.0.0
Finished processing dependencies for pip==9.0.0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[xavfernandez]

Apparently it globally works, it just makes some noise during the installation.
No urgency here 😊

[dstufft]

Ok. I was SUPER confused how none of our tests or manual testing caught this, but if it works it just poops out a message with -v that makes sense.

Sent from my iPhone

On Nov 2, 2016, at 7:11 PM, Xavier Fernandez [email protected] wrote:

Apparently it globally works, it just makes some noise during the installation.
No urgency here 😊

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[rudyvogt]

@dstufft
yes it seems to work after install - looks like a bit of noise. I could install other packages/upgrade
i have python 2.7.8 installed

Pip still works after this error yes? Or is it completely broken?

[gukoff]

Calling python2.7 -mcompileall on the virtualenv folder fails with an error because of this.
Perhaps, compiling pip should become a test.

[jmatt]

This problem is breaking conda-build (the library which builds conda recipes). So it's more than just noise. I'm not sure exactly why but I suspect it's calling -mcompileall. I'm looking forward to 9.0.1.

[rudyvogt]

Just tested 9.0.1 seems to work 100% - Great work for fixing it so quickly

[jamshid]

Seeing this error simply installing pip using easy_install on centos 6.9.

$ docker run -ti centos:6.9 bash
Unable to find image 'centos:6.9' locally
6.9: Pulling from library/centos
993c50d47469: Pull complete 
Digest: sha256:5cf988fbf143af398f879bd626ee677da3f8d229049b7210790928a02613ab26
Status: Downloaded newer image for centos:6.9
[root@b3dbcef03a1b /]# 
[root@b3dbcef03a1b /]# yum install -y python-setuptools && easy_install pip
Loaded plugins: fastestmirror, ovl
Setting up Install Process
base                                                                                                                 | 3.7 kB     00:00     
base/primary_db                                                                                                      | 4.7 MB     00:01     
extras                                                                                                               | 3.4 kB     00:00     
extras/primary_db                                                                                                    |  30 kB     00:00     
updates                                                                                                              | 3.4 kB     00:00     
updates/primary_db                                                                                                   | 8.1 MB     00:03     
Resolving Dependencies
--> Running transaction check
---> Package python-setuptools.noarch 0:0.6.10-4el6_9 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================
 Package                                Arch                        Version                              Repository                    Size
============================================================================================================================================
Installing:
 python-setuptools                      noarch                      0.6.10-4el6_9                        updates                      336 k

Transaction Summary
============================================================================================================================================
Install       1 Package(s)

Total download size: 336 k
Installed size: 1.5 M
Downloading Packages:
python-setuptools-0.6.10-4el6_9.noarch.rpm                                                                           | 336 kB     00:00     
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Importing GPG key 0xC105B9DE:
 Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <[email protected]>
 Package: centos-release-6-9.el6.12.3.x86_64 (@CentOS/6.9)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : python-setuptools-0.6.10-4el6_9.noarch                                                                                   1/1 
  Verifying  : python-setuptools-0.6.10-4el6_9.noarch                                                                                   1/1 

Installed:
  python-setuptools.noarch 0:0.6.10-4el6_9                                                                                                  

Complete!
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 10.0.1
Downloading https://files.pythonhosted.org/packages/ae/e8/2340d46ecadb1692a1e455f13f75e596d4eab3d11a57446f08259dee8f02/pip-10.0.1.tar.gz#sha256=f2bd08e0cd1b06e10218feaf6fef299f473ba706582eb3bd9d52203fdbd7ee68
Processing pip-10.0.1.tar.gz
Running pip-10.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-WSSKBu/pip-10.0.1/egg-dist-tmp-YWu5LC
/usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching '.landscape.yml'
warning: no previously-included files found matching 'src/pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching '*-requirements.txt'
warning: no previously-included files found matching 'appveyor.yml'
warning: no previously-included files found matching 'src/pip/_vendor/six'
warning: no previously-included files matching '*.pyi' found under directory 'src/pip/_vendor'
no previously-included directories found matching '.github'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'news'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/pep425tags.py', 84, 35, "    if not soabi and impl in {'cp', 'pp'} and hasattr(sys, 'maxunicode'):\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/baseparser.py', 156, 37, '        section_items = {name: [] for name in override_order}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/utils/misc.py', 849, 29, '    reverse = {value: key for key, value in enums.items()}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/resolve.py', 35, 35, '    _allowed_strategies = {"eager", "only-if-needed", "to-satisfy-only"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/index.py', 526, 43, '                        {str(c.version) for c in all_candidates},\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/cache.py', 37, 35, '        _valid_formats = {"source", "binary"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/configuration.py', 101, 27, '            variant: [] for variant in self._override_order\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/wheel.py', 173, 31, '        parent_dir: scripts for parent_dir, scripts in grouped_by_dir.items()\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/basecommand.py', 195, 21, '                } for name in [\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/list.py', 170, 23, '        return {pkg for pkg in packages if pkg.key not in dep_keys}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/__init__.py', 42, 30, 'commands_dict = {c.name: c for c in commands_order}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/freeze.py', 11, 18, "DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel'}\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/commands/download.py', 142, 52, "        binary_only = FormatControl(set(), {':all:'})\n"))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/req/req_uninstall.py', 137, 37, '        os.path.join(folder, "*") for folder in folders\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/req/req_install.py', 98, 51, '                pkg_resources.safe_extra(extra) for extra in req.extras\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_internal/compat.py', 180, 24, 'stdlib_pkgs = {"python", "wsgiref", "argparse"}\n'))

SyntaxError: ('invalid syntax', ('build/bdist.linux-x86_64/egg/pip/_vendor/html5lib/_trie/_base.py', 16, 21, '        return {x for x in keys if x.startswith(prefix)}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/pep425tags.py', 84, 35, "    if not soabi and impl in {'cp', 'pp'} and hasattr(sys, 'maxunicode'):\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/baseparser.py', 156, 37, '        section_items = {name: [] for name in override_order}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/resolve.py', 35, 35, '    _allowed_strategies = {"eager", "only-if-needed", "to-satisfy-only"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/index.py', 526, 43, '                        {str(c.version) for c in all_candidates},\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/cache.py', 37, 35, '        _valid_formats = {"source", "binary"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/configuration.py', 101, 27, '            variant: [] for variant in self._override_order\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/wheel.py', 173, 31, '        parent_dir: scripts for parent_dir, scripts in grouped_by_dir.items()\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/basecommand.py', 195, 21, '                } for name in [\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/compat.py', 180, 24, 'stdlib_pkgs = {"python", "wsgiref", "argparse"}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/utils/misc.py', 849, 29, '    reverse = {value: key for key, value in enums.items()}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/list.py', 170, 23, '        return {pkg for pkg in packages if pkg.key not in dep_keys}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/__init__.py', 42, 30, 'commands_dict = {c.name: c for c in commands_order}\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/freeze.py', 11, 18, "DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel'}\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/commands/download.py', 142, 52, "        binary_only = FormatControl(set(), {':all:'})\n"))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/req/req_uninstall.py', 137, 37, '        os.path.join(folder, "*") for folder in folders\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_internal/req/req_install.py', 98, 51, '                pkg_resources.safe_extra(extra) for extra in req.extras\n'))

SyntaxError: ('invalid syntax', ('/usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg/pip/_vendor/html5lib/_trie/_base.py', 16, 21, '        return {x for x in keys if x.startswith(prefix)}\n'))

Adding pip 10.0.1 to easy-install.pth file
Installing pip script to /usr/bin
Installing pip2.6 script to /usr/bin
Installing pip2 script to /usr/bin

Installed /usr/lib/python2.6/site-packages/pip-10.0.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip

[pradyunsg]

pip 10 dropped support for Python 2.6. You can install pip 9 (current latest 9.0.3) if you want to use it on Python 2.6.

Further, you seem to be coming from a really old version of setuptools.

If you want to install pip on Python 2.6, one way is to use https://bootstrap.pypa.io/2.6/get-pip.py .

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.