Salt: Salt 2015.8.5 incompatible with Pip v8.1.2

Created on 11 May 2016 · 18Comments · Source: saltstack/salt

Description of Issue/Question

Pip 8.1.2 was just released and the poorly timed upgrade to it broke all of our salt states that rely on pip.
Force downgrading and locking pip to version 8.1.1 resolves the issue for now.

Setup

admin@jay-dev-001:~$ pip --version
pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

    config['upgrade-pip'] = {
        'pip.installed': [
            {'name': 'pip'},
            {'reload_modules': True},
            {'upgrade': True},
            {'require': [{'pkg': 'core_packages'}]}
        ]
    }

    config['upgrade-requests'] = {
        'pip.installed': [
            {'name': 'requests'},
            {'reload_modules': True},
            {'upgrade': True},
            {'require': [
                {'pkg': 'core_packages'},
                {'pip': 'pip'}
            ]}
        ]
    }

    config['docker-py'] = {
        'pip.installed': [
            {'name': 'docker-py == 1.4.0'},
            {'reload_modules': True},
            {'require': [
                {'pip': 'requests'},
                {'pip': 'pip'}
            ]}
        ]
    }

Steps to Reproduce Issue

salt * state.apply

----------
          ID: upgrade-pip
    Function: pip.installed
        Name: pip
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1491, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 592, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 176, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 02:02:36.041407
    Duration: 55.199 ms
     Changes:
----------
          ID: upgrade-requests
    Function: pip.installed
        Name: requests
      Result: False
     Comment: One or more requisite failed: core.docker_setup.upgrade-pip
     Started:
    Duration:
     Changes:
----------
          ID: install-python-requests
    Function: pip.installed
        Name: requests
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1491, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 592, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 176, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 02:02:36.484286
    Duration: 56.197 ms
     Changes:
----------
          ID: docker-py
    Function: pip.installed
        Name: docker-py == 1.4.0
      Result: False
     Comment: One or more requisite failed: core.docker_setup.upgrade-pip, core.docker_setup.upgrade-requests, consul.install.install-python-requests
     Started:
    Duration:
     Changes:
----------
          ID: install-python-consul
    Function: pip.installed
        Name: python-consul
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1491, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 592, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 176, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 02:02:46.170152
    Duration: 58.18 ms
     Changes:

Versions Report

admin@saltshaker-001:~$ salt --versions-report
Salt Version:
           Salt: 2015.8.5

Dependency Versions:
         Jinja2: 2.7.3
       M2Crypto: Not Installed
           Mako: 1.0.0
         PyYAML: 3.11
          PyZMQ: 14.4.0
         Python: 2.7.9 (default, Mar  1 2015, 12:57:24)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.2
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: 0.3.6
          smmap: 0.8.2
        timelib: Not Installed

System Versions:
           dist: debian 8.3
        machine: x86_64
        release: 3.16.0-4-amd64
         system: debian 8.3

Execution Module P1 Platform State Module Upstream Bug fixed-pending-your-verification severity-medium

Source

jaybocc2

👍15

Most helpful comment

Downgrading Pip with pip install pip==8.1.1 seemed to fix it for now.

lietu on 12 May 2016

👍5 🎉1

All 18 comments

can verify this as well, same error

uacfboblanton on 11 May 2016

Affects 2015.8.8.2 as well.

codekoala on 11 May 2016

Can confirm downgrading to pip 8.1.1 for Salt 2015.8.8.2 works.

jamesp9 on 11 May 2016

same error here with 8.1.2, downgrade to 8.1.1 works fine

tom-aa on 11 May 2016

This affects the 'develop' branch too. Downgrading pip works, but the virtualenv state module is not working for me anymore (perhaps unrelated...).

rrei on 11 May 2016

Thanks everyone for reporting. This has affected us as well and we're looking into it. As @kiorky suggests, this seems to be an upstream bug.

jfindlay on 11 May 2016

Yes, and for us it affects number of projects, unrelated to salt in any way...
And upstream doesnt want to fix for now :gun:

kiorky on 11 May 2016

Upstream says that they don't support this api being stable, so I am working on a patch

thatch45 on 11 May 2016

basically this goes all the way up to pkg_resources, which I will admit, is the main reason I insist on the standard packages of Salt being built with distutils when possible, since I have seen a lot of breakage in pkg_resources over the years

thatch45 on 11 May 2016

ok, this PR should fix it, and this fix will come out in the releases we are working on right now:
https://github.com/saltstack/salt/pull/33180

thatch45 on 11 May 2016

Downgrading Pip with pip install pip==8.1.1 seemed to fix it for now.

lietu on 12 May 2016

👍5 🎉1

We should have this all wrapped up with the merge of #33180. I'm going to go ahead and close this.

cachedout on 13 May 2016

FYI, this also shows up in Ubuntu Xenial package '8.1.1-2ubuntu0.1' even though "pip --version" returns "8.1.1". Installing the older version "python-pip=8.1.1-2" works.

moloney on 26 May 2016

👍3

This is still broken on Ubuntu 16.04
suggested fix no work
The true code fix is here: https://github.com/saltstack/salt/pull/33180/files

if one puts the below in a file called file (including greater than and hyphens and 23a24):



23a24

> import re

176,177c177,182

<         ret['prefix'] = install_req.req.project_name

<         ret['version_spec'] = install_req.req.specs

    ret['prefix'] = re.sub('[^A-Za-z0-9.]+', '-', install_req.name)
    if hasattr(install_req, "specifier"):
        specifier = install_req.specifier
    else:
        specifier = install_req.req.specifier
    ret['version_spec'] = [(spec.operator, spec.version) for spec in specifier]

and does
sudo patch /usr/lib/python2.7/dist-packages/salt/states/pip_state.py file

The problem goes away

haraldrudell on 5 Aug 2016

👍3

This is still broken in 2015.8.12

moloney on 1 Oct 2016

I was still experiencing this issue with Ubuntu's python-pip (8.1.1). I had to run pip install pip==8.1.1 --force --upgrade to fix the issue.

bernielomax on 5 Apr 2017

👍3 ❤1

So is the idea that on ubuntu one needs to first install the python-pip apt package and then reinstall pip with pip?

brentsmyth on 7 Jun 2017

I have the same problem with pip 9.0.1. It only fails with salt-minion 2015.8.8, but salt-minion 2015.8.13 works. The recipe is extremely simple:

numpy:
  pip.installed

host:
----------
          ID: numpy
    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1626, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 592, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 176, in _check_pkg_version_format
                  ret['prefix'] = install_req.req.project_name
              AttributeError: 'Requirement' object has no attribute 'project_name'
     Started: 13:48:35.969465
    Duration: 2.269 ms
     Changes: