pip 10.0.0b2 check false positives about some packages

[mgedmin]

• Pip version: 10.0.0b2
• Python version: 2.7.14
• Operating system: Ubuntu 17.10

Description:

I have a virtualenv in ~/.venv/ that was created with --system-site-packages and is on my $PATH so that I can pip install various tools (like py.test, or flake8, or ansible) and have them readily available for me.
The --system-site-packages is so that I can avoid slow compilations by installing things like python-lxml, python-pillow, and python-gobject (the latter of which was not available on PyPI at all until recently).

To make things more interesting I sometimes use pip install --user as well (the actual reason I do that is that I need to have flake8 in ~/.local/lib/python3.x/site-packages, or I can't lint Python-3-only packages, due to the way flake8 relies on the ast module).

I ran pip check for a lark after upgrading pip to 10.0.0b2 and was surprised by this:

$ pip check
acme 0.22.2 requires mock, which is not installed.
acme 0.22.2 requires pyrfc3339, which is not installed.
acme 0.22.2 requires pytz, which is not installed.
certbot-apache 0.22.2 requires mock, which is not installed.
pytest 3.5.0 requires funcsigs, which is not installed.
ansible-cmdb 1.25 requires mako, which is not installed.
requests 2.18.4 requires chardet, which is not installed.
imgdiff 1.6.0 requires pillow, which is not installed.
certbot 0.22.2 requires configobj, which is not installed.
certbot 0.22.2 requires mock, which is not installed.
certbot 0.22.2 requires parsedatetime, which is not installed.
certbot 0.22.2 requires pyrfc3339, which is not installed.
certbot 0.22.2 requires pytz, which is not installed.

The supposedly-missing packages _are_ available (e.g. mock appears to be in ~/.local/lib/python2.7/site-packages/mock/), and pip itself _knows_ about them:

$ pip list | grep mock
mock                         2.0.0        

Besides, I just did a pip install -U certbot-apache and it upgraded my certbot from 0.19 to 0.22.2, without trying to (re)install mock, while simultaneously complaining that mock is not installed!

Collecting certbot-apache
  Downloading certbot_apache-0.22.2-py2.py3-none-any.whl (218kB)
    100% |████████████████████████████████| 225kB 1.5MB/s 
Requirement not upgraded as not directly required: zope.interface in ./.venv/lib/python2.7/site-packages (from certbot-apache) (4.4.3)
Requirement not upgraded as not directly required: python-augeas in ./.venv/lib/python2.7/site-packages (from certbot-apache) (1.0.3)
Requirement not upgraded as not directly required: setuptools in ./.venv/lib/python2.7/site-packages (from certbot-apache) (39.0.1)
Collecting certbot>=0.21.1 (from certbot-apache)
  Downloading certbot-0.22.2-py2.py3-none-any.whl (284kB)
    100% |████████████████████████████████| 286kB 1.7MB/s 
Collecting acme>=0.21.1 (from certbot-apache)
  Downloading acme-0.22.2-py2.py3-none-any.whl (75kB)
    100% |████████████████████████████████| 81kB 1.3MB/s 
Requirement not upgraded as not directly required: zope.component in ./.venv/lib/python2.7/site-packages (from certbot-apache) (4.4.1)
Requirement not upgraded as not directly required: mock in ./.local/lib/python2.7/site-packages (from certbot-apache) (2.0.0)
Requirement not upgraded as not directly required: cffi>=1.0.0 in ./.venv/lib/python2.7/site-packages (from python-augeas->certbot-apache) (1.11.5)
Requirement not upgraded as not directly required: pyrfc3339 in /usr/lib/python2.7/dist-packages (from certbot>=0.21.1->certbot-apache) (1.0)
Requirement not upgraded as not directly required: configobj in /usr/lib/python2.7/dist-packages (from certbot>=0.21.1->certbot-apache) (5.0.6)
Requirement not upgraded as not directly required: ConfigArgParse>=0.9.3 in ./.venv/lib/python2.7/site-packages (from certbot>=0.21.1->certbot-apache) (0.12.0)
Requirement not upgraded as not directly required: pytz in /usr/lib/python2.7/dist-packages (from certbot>=0.21.1->certbot-apache) (2017.2)
Collecting josepy (from certbot>=0.21.1->certbot-apache)
  Downloading josepy-1.0.1-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 2.1MB/s 
Requirement not upgraded as not directly required: cryptography>=1.2 in ./.venv/lib/python2.7/site-packages (from certbot>=0.21.1->certbot-apache) (2.2.2)
Requirement not upgraded as not directly required: parsedatetime>=1.3 in /usr/lib/python2.7/dist-packages (from certbot>=0.21.1->certbot-apache) (2.4)
Requirement not upgraded as not directly required: requests[security]>=2.4.1 in ./.venv/lib/python2.7/site-packages (from acme>=0.21.1->certbot-apache) (2.18.4)
Requirement not upgraded as not directly required: PyOpenSSL>=0.13 in ./.venv/lib/python2.7/site-packages (from acme>=0.21.1->certbot-apache) (17.5.0)
Requirement not upgraded as not directly required: six>=1.9.0 in ./.venv/lib/python2.7/site-packages (from acme>=0.21.1->certbot-apache) (1.11.0)
Requirement not upgraded as not directly required: zope.event in ./.venv/lib/python2.7/site-packages (from zope.component->certbot-apache) (4.3.0)
Requirement not upgraded as not directly required: funcsigs>=1; python_version < "3.3" in ./.local/lib/python2.7/site-packages (from mock->certbot-apache) (1.0.2)
Requirement not upgraded as not directly required: pbr>=0.11 in ./.venv/lib/python2.7/site-packages (from mock->certbot-apache) (3.1.1)
Requirement not upgraded as not directly required: pycparser in ./.venv/lib/python2.7/site-packages (from cffi>=1.0.0->python-augeas->certbot-apache) (2.18)
Requirement not upgraded as not directly required: enum34; python_version < "3" in ./.venv/lib/python2.7/site-packages (from cryptography>=1.2->certbot>=0.21.1->certbot-apache) (1.1.6)
Requirement not upgraded as not directly required: idna>=2.1 in ./.venv/lib/python2.7/site-packages (from cryptography>=1.2->certbot>=0.21.1->certbot-apache) (2.6)
Requirement not upgraded as not directly required: asn1crypto>=0.21.0 in ./.venv/lib/python2.7/site-packages (from cryptography>=1.2->certbot>=0.21.1->certbot-apache) (0.24.0)
Requirement not upgraded as not directly required: ipaddress; python_version < "3" in ./.venv/lib/python2.7/site-packages (from cryptography>=1.2->certbot>=0.21.1->certbot-apache) (1.0.19)
Requirement not upgraded as not directly required: urllib3<1.23,>=1.21.1 in ./.venv/lib/python2.7/site-packages (from requests[security]>=2.4.1->acme>=0.21.1->certbot-apache) (1.22)
Requirement not upgraded as not directly required: certifi>=2017.4.17 in ./.venv/lib/python2.7/site-packages (from requests[security]>=2.4.1->acme>=0.21.1->certbot-apache) (2018.1.18)
Requirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/dist-packages (from requests[security]>=2.4.1->acme>=0.21.1->certbot-apache) (3.0.4)
acme 0.22.2 requires mock, which is not installed.
acme 0.22.2 requires pyrfc3339, which is not installed.
acme 0.22.2 requires pytz, which is not installed.
certbot-apache 0.22.2 requires mock, which is not installed.
pytest 3.5.0 requires funcsigs; python_version < "3.0", which is not installed.
ansible-cmdb 1.25 requires mako>=1.0, which is not installed.
requests 2.18.4 requires chardet<3.1.0,>=3.0.2, which is not installed.
imgdiff 1.6.0 requires Pillow, which is not installed.
certbot 0.22.2 requires configobj, which is not installed.
certbot 0.22.2 requires mock, which is not installed.
certbot 0.22.2 requires parsedatetime>=1.3, which is not installed.
certbot 0.22.2 requires pyrfc3339, which is not installed.
certbot 0.22.2 requires pytz, which is not installed.
bindep 2.1.0 has requirement pbr!=0.7,<1.0,>=0.6, but you'll have pbr 3.1.1 which is incompatible.
Installing collected packages: josepy, acme, certbot, certbot-apache
  Found existing installation: acme 0.19.0
    Uninstalling acme-0.19.0:
      Successfully uninstalled acme-0.19.0
  Found existing installation: certbot 0.19.0
    Uninstalling certbot-0.19.0:
      Successfully uninstalled certbot-0.19.0
  Found existing installation: certbot-apache 0.19.0
    Uninstalling certbot-apache-0.19.0:
      Successfully uninstalled certbot-apache-0.19.0
Successfully installed acme-0.22.2 certbot-0.22.2 certbot-apache-0.22.2 josepy-1.0.1

[cjerdonek]

If you downgrade pip, does pip check then start working? (This isn't clear from your message.)

[mgedmin]

pip check with pip 9.0.3 does not complain about mock or chardet or funcsigs.

(It does complain about "launchpadlib 1.10.5 requires testresources, which is not installed.", which seems correct: I don't have testresources installed. I have launchpadlib from the Ubuntu package, which does not declare a dependency on testresources and seems to work anyway. Perhaps it's a test-only dependency.)

[pradyunsg]

Thanks for testing out the beta @mgedmin! :)

I know what happened here. Working on a fix.

[pradyunsg]

Fixed in the release branch. Thanks again @mgedmin for reporting this. :)

If you could confirm that this is fixed on pip install -U https://github.com/pypa/pip/archive/release/10.0.0.zip, that would be awesome.

[mgedmin]

It is fixed in pip 10.0.0b2 from your URL:

$ pip install -U https://github.com/pypa/pip/archive/release/10.0.0.zip
...
$ pip --version
pip 10.0.0b2 from /home/mg/.venv/local/lib/python2.7/site-packages/pip (python 2.7)

$ pip check
launchpadlib 1.10.5 requires testresources, which is not installed.

[pradyunsg]

Great! Thanks for confirming @mgedmin.

I'll close this issue now.

[mgedmin]

(EDIT: accidentally commented on the wrong bug, now moved the comment where it belongs.)

[mgedmin]

Oops, I commented on the wrong bug! Sorry about that.

[crestenstclair]

If people are experiencing this in Ubuntu and are unclear on the answer, installing the dependency:

sudo apt install python-testresources

Will fix the issue.

[Kipngetich33]

I have tried the method above but it's not working I am getting this error:
E: Unable to locate package python-test resources

[treedbox]

@honkycat @Kipngetich33 sudo apt install python-testresources works fine, just close the terminal and open it again.

[Wilsonlin06]

I also have this issue "launchpadlib 1.10.6 requires testresources, which is not installed."
I did sudo apt install python-testresources, but the issue is still there.
Can someone help me with this please?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.