pip3 list -o fails with TypeError: unorderable types: Version() > SetuptoolsVersion()

[jlec]

In case vendor packages of pip are unbundled but the vendored "packaging" in setuptool (pkg_resources) not, pip3 list -o fails with

Exception:
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/usr/lib64/python3.4/site-packages/pip/commands/list.py", line 95, in run
    self.run_outdated(options)
  File "/usr/lib64/python3.4/site-packages/pip/commands/list.py", line 107, in run_outdated
    if version > dist.parsed_version:
TypeError: unorderable types: Version() > SetuptoolsVersion()

I cannot find the origin of the problem. packaging is 15.3 as it is in setuptools-18.2 and pip-7.1.2.

[jlec]

Python2 aka. pip2 is fine

[dstufft]

I don't think there's anything we can reasonably do here, you're running a patched or unsupported version of pip. You should file a bug with whomever gave you your copy of pip and ask them to stop patching it.

[jlec]

pip is providing the possibility to unbundle the _vendor packages, doesn't it? So it is supported. That is what _I_ did as the downstream package maintainer. Due to the circular dependency of setuptools without _vendor and packaging, we sadly cannot do this for setuptools as well.

Something reasonable to do would be to help me find out what in the intersection of setuptools, packaging and pip is not python3 compatible.

[dstufft]

The answer to that is a little complicated, but essentially no we don't. Currently the only way to get a version of pip without a bundled _vendor is to manually patch pip. For a little while (added in 6.0, removed in 7.1) we did offer an environment variable you could set to unbundle it but even when we did, actually using that environment variable printed out a statement that it was unsupported and only existed for downstream redistributors who are going to be patching pip anyways. What little support we have for unbundling is essentially just there as a favor to downstreams who are going to unbundle to try and minimize the diff they have to cary against pip, but any support we provide is at most best effort.

I see now that you're a downstream developer, I didn't realize that originally. It's not uncommon for downstream to unbundle things, that to cause problems for end users, and then end users come to us asking for support. I generally redirect those users back to the people who modified their copy of pip since it was their modifications that caused the problems in the first place. I would much rather that downstream's _didn't_ unbundle because pip is in a fairly unique situation that I feel warrants the bundling (hence why we've done it, I feel dirty for doing it but it's basically a "least worst" solution).

Basically, your problem is that Python 3 is stricter about comparisons and requires explicit opt in to have classes be comparable. The problem is that while SetuptoolsVersion is a subclass of Version (and thus, typically fine to be compared to each other) Python sees pkg_resources._vendor.packaging.version:Version as an entirely different module/class from packaging.version:Version and has no idea they can be compared. I don't think that can be solved in a particularly great way since it would require patching both setuptools and packaging I think and if one of those got upgraded outside of the distro package manager it would break.

However, the packaging library shouldn't depend on setuptools. I was careful to only import things from itself or the standard library and that library isn't allowed to have dependencies. The only place it uses setuptools is in it's setup.py, but that's only build time not run time. If that's what is causing your circular dependency, the setup.py can be switched to rely on just plain distutils because It's not using any setuptools specific functionality.

[jlec]

Thanks Donald for that lengthy explanation on all those matters!

I realized that pip isn't actively promoting the unbundling. Even the README still contains the old envvar solution and doesn't mention the new patching version of unbundling. I understand your concerns about that modification done by downstream and about the special requirements of pip, but as you are not modifying the bundled libs except for the import compatibility I would say it is worth some investigation, if it would work here.

That makes me come to the last point. The circular dep is only triggered by the buildtime dependency on setuptools. So if it would be possible to have packaging not depend on setuptools at all, we would solve the issue.

Thanks for you lesson on the namesspaces coming from different import paths in python. I will do some reading on that matter.

[dstufft]

I'm going to go ahead and close this issue, I've opened pypa/packaging#38 which should solve it.

[jlec]

setuptools-19.4 break this again.

[jlec]

Still happens with 19.6. Any idea what the problem is?

[xavfernandez]

@jlec with which pip version ?

$ mktmpenv -p /usr/bin/python3.4
Running virtualenv with interpreter /usr/bin/python3.4
Using base prefix '/usr'
New python executable in 34c0328e23769b53/bin/python3.4
Also creating executable in 34c0328e23769b53/bin/python
Installing setuptools, pip...done.
...
(34c0328e23769b53)$ pip install setuptools -U
...
Successfully installed setuptools-19.6
(34c0328e23769b53)$ pip list
pip (8.0.2)
setuptools (19.6)
(34c0328e23769b53)$ pip install django==1.4
....
(34c0328e23769b53)$ pip list -o
Django (1.4) - Latest: 1.9.1 [wheel]

Edited: sorry, I didn't read carefully, there is some debundling that I won't try to reproduce.

[jlec]

I am testing now 8.0.2

[sherpya]

this bug is still present in 8.1.2 but fixed in master, please make a bugfix release

[moon-chilled]

Workaround for anyone experiencing this:

# pip3 install --force pip

To forcibly install the "real" version of pip from PiPy.

[Wtower]

The same happens for me with 9.0.1. Solved with:

pip install --force pip==9.0.0

[Kami-no]

Final solution:

pip install --force pip==9.0.0
pip install -U pip

Suppose smth wrong with pip from distro (Kali Linux + virtualenv)

[alexagrf]

Still happening here on pip 9.0.1

[Kami-no]

It's not about pip version it's about pip from disto. Force to download pip from pip by downgrading it (even any lesser release). Now the problem is solved (pip3 list -o) and you're free to upgrade pip or any other package with pip.

[pradyunsg]

As @Kami-no said, this is caused by the fact that the way some distros package pip is not supported and fragile. The way to fix this to install pip from PyPI. You can even do this to fix it:

pip install --ignore-installed pip

I recommend passing --user additionally.

[alexagrf]

Thanks @pradyunsg. I should learn to read. (I also hadn't realised that the pip in a virtualenv would be just a copy of the distribution's pip, but yeah, where else is it going to come from otherwise)

[gleb-chipiga]

• Ubuntu 17.10
• Python 3.6.3
• Pip 9.0.1

pip3 list -o
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
Exception:
Traceback (most recent call last):
  File "/home/gleb/.virtualenvs/dev/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/gleb/.virtualenvs/dev/lib/python3.6/site-packages/pip/commands/list.py", line 157, in run
    packages = self.get_outdated(packages, options)
  File "/home/gleb/.virtualenvs/dev/lib/python3.6/site-packages/pip/commands/list.py", line 168, in get_outdated
    dist for dist in self.iter_packages_latest_infos(packages, options)
  File "/home/gleb/.virtualenvs/dev/lib/python3.6/site-packages/pip/commands/list.py", line 169, in <listcomp>
    if dist.latest_version > dist.parsed_version
TypeError: '>' not supported between instances of 'Version' and 'SetuptoolsVersion'

The error causes the pip-review to crash jgonggrijp/pip-review#60

[E3V3A]

I can confirm this is still a problem. I had all the latest versions, but:

$ pip3 list --outdated
Exception:
...
TypeError: '>' not supported between instances of 'Version' and 'SetuptoolsVersion'

But many thanks to @pradyunsg because this:

pip3 install --ignore-installed pip --user

Fixed it! ❤️

[SilkAndSlug]

That command, pip3 install --ignore-installed pip --user, broke pip3 for me

: pip3 list -o
Traceback (most recent call last):
  File "/usr/bin/pip3", line 9, in <module>
    from pip import main
ImportError: cannot import name 'main'

[avma]

Linux mint 19 Tara
Python 3.6.5
Pip 9.0.1

$ pip3 list --outdated
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/list.py", line 157, in run
    packages = self.get_outdated(packages, options)
  File "/usr/lib/python3/dist-packages/pip/commands/list.py", line 168, in get_outdated
    dist for dist in self.iter_packages_latest_infos(packages, options)
  File "/usr/lib/python3/dist-packages/pip/commands/list.py", line 169, in <listcomp>
    if dist.latest_version > dist.parsed_version
**TypeError: '>' not supported between instances of 'Version' and 'Version'**

[git001]

• MINGW64_NT-6.1 aleks-PC 2.11.1(0.329/5/3) 2018-09-10 14:19 x86_64 Msys
• Python 3.7.0
• pip 18.0 from D:/APPS/msys64/mingw64/lib/python3.7/site-packages/pip (python 3.7)

$ pip3 list --outdated
Exception:
Traceback (most recent call last):
  File "D:/APPS/msys64/mingw64/lib/python3.7/site-packages\pip\_internal\basecommand.py", line 141, in main
    status = self.run(options, args)
  File "D:/APPS/msys64/mingw64/lib/python3.7/site-packages\pip\_internal\commands\list.py", line 136, in run
    packages = self.get_outdated(packages, options)
  File "D:/APPS/msys64/mingw64/lib/python3.7/site-packages\pip\_internal\commands\list.py", line 147, in get_outdated
    dist for dist in self.iter_packages_latest_infos(packages, options)
  File "D:/APPS/msys64/mingw64/lib/python3.7/site-packages\pip\_internal\commands\list.py", line 148, in <listcomp>
    if dist.latest_version > dist.parsed_version
TypeError: '>' not supported between instances of 'Version' and 'Version'

[PeterMosmans]

Hi @git001 - It seems like you're using MSYS2... Apparently (?) this is fixed by version 18.1:

$ python3 -m pip install -U pip
$ pip3 list --outdated 1>/dev/null; echo $?

The last command returns 0 :)

[git001]

@PeterMosmans Thanks it works now.

aleks@aleks-PC MINGW64 ~
$ python3 -m pip install -U pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
    100% |################################| 1.3MB 186kB/s
Installing collected packages: pip
  Found existing installation: pip 18.0
    Uninstalling pip-18.0:
      Successfully uninstalled pip-18.0
Successfully installed pip-18.1

aleks@aleks-PC MINGW64 ~
$ pip3 list --outdated 1>/dev/null; echo $?
0

[avma]

Hi Peter,

I ran: $ python3 -m pip install -U pip

and now I'm getting a new error:
$ pip3 list --outdated Traceback (most recent call last): File "/usr/bin/pip3", line 9, in <module> from pip import main ImportError: cannot import name 'main'

The 2nd command returns 1...

It seems PIP3 is not working anymore but PIP now is hooked to python3 (it used to run python 2.7 before).
$ pip -V pip 18.1 from /home/avi/.local/lib/python3.6/site-packages/pip (python 3.6)

Is there a way to fix this?
My Regards
Avi

[SilkAndSlug]

@avma : in my case, I had multiple copies of pip installed. The user's copy was broken, but the upgrade changed the system copy. Deleting the user's copy fixed the error.

To find your user's copy:

python3
import sys
print(sys.path)
[Ctrl-D]

You'll get a long list, in the middle of which will be something like /home/avi/.local/lib/python3.6/site-packages. If not, change the following to suit!

Next disable the user's package by renaming it:

cd /home/avi/.local/lib/python3.6/site-packages
mv pip pip.bak

(Almost) finally, test if that fixed anything:

pip3 -V

Hopefully you'll get something like pip 18.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.5)

Finally, test (pip3 list --outdated) and tidy (rm -rf pip.bak) or revert (mv pip.bak pip).

[benoit-pierre]

You can't break the system pip by installing another version in your site package. However the pip's main entrypoint changed in newer versions, and even when you use one of the /usr/bin/pip launcher, you're user installed version will be used (that's normal, that's how the Python path work): hence the import error. You can use python -s -m pip to invoke the system version even if you have another user installed version (-s will disable the user site). Finally, you can use python -m site to find out the location of user site packages.

[avma]

It looks like both python 3.6 and 2.7 are installed and available on my system (mint 19 tara), this is ok.

$ python -m site
sys.path = [
'/home/avi',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/home/avi/.local/lib/python2.7/site-packages',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
]
USER_BASE: '/home/avi/.local' (exists)
USER_SITE: '/home/avi/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: True

$ python3 -m site
sys.path = [
'/home/avi',
'/usr/lib/python36.zip',
'/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload',
'/home/avi/.local/lib/python3.6/site-packages',
'/usr/local/lib/python3.6/dist-packages',
'/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/avi/.local' (exists)
USER_SITE: '/home/avi/.local/lib/python3.6/site-packages' (exists)
ENABLE_USER_SITE: True

However, both pip and pip3 cli invokes python 3.6. Before, pip invoked python 2.7 and pip3 invoked python 3.6.

$ pip3 -V
pip 18.1 from /home/avi/.local/lib/python3.6/site-packages/pip (python 3.6)

$ pip -V
pip 18.1 from /home/avi/.local/lib/python3.6/site-packages/pip (python 3.6)

Is it possible to get it back working that way?

[Dmole]

Just remove the --outdated option if it's not going to work on most of the real world systems.

pip3 -V
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)

[PeterMosmans]

$ pip3 -V
pip 18.1 from /home/avi/.local/lib/python3.6/site-packages/pip (python 3.6)

$ pip -V
pip 18.1 from /home/avi/.local/lib/python3.6/site-packages/pip (python 3.6)

Is it possible to get it back working that way?

Use python -m pip and python3 -m pip: This is also the preferred way to invoke pip, with the -m flag

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.