Salt: pip state broken in 2015.8.9 with pip <6.0

[tmehlinger]

Description

pip state fails with AttributeError in Salt 2015.8.9 with pip < 6.0

Setup

Ubuntu Trusty
python-pip installed from mainline apt (pip 1.5.4)

Steps to Reproduce Issue

Run a state with pip.installed in it with pip 1.5.4 installed on the system.

Versions Report

# salt-call --versions-report
Salt Version:
           Salt: 2015.8.9

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: 0.9.1
         PyYAML: 3.10
          PyZMQ: 14.0.1
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 4.2.0-35-generic
         system: Ubuntu 14.04 trusty

Stack trace from the minion output:

    Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 597, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 181, in _check_pkg_version_format
                  specifier = install_req.req.specifier
              AttributeError: Requirement instance has no attribute 'specifier'
     Started: 20:19:45.883326
    Duration: 3.61 ms
     Changes:

Relevant code in pip: https://github.com/pypa/pip/blob/1.5.6/pip/req.py#L170

As you can see, req is a string.

I posted all this as comments on #33180, then realized I should open an issue. Sorry about that.

[shawnbutts]

I'm also seeing something simalar with pip 6.1.1

Function: pip.installed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.6/site-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.6/site-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
                  self.gen.throw(type, value, traceback)
                File "/usr/lib/python2.6/site-packages/salt/utils/context.py", line 42, in func_globals_inject
                  yield
                File "/usr/lib/python2.6/site-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.6/site-packages/salt/states/pip_state.py", line 597, in installed
                  out = _check_pkg_version_format(pkg)
                File "/usr/lib/python2.6/site-packages/salt/states/pip_state.py", line 182, in _check_pkg_version_format
                  ret['version_spec'] = [(spec.operator, spec.version) for spec in specifier]
              TypeError: 'SpecifierSet' object is not iterable
     Started: 20:39:00.133249
    Duration: 0.807 ms
     Changes:

[shawnbutts]

I have confirmed that reverting https://github.com/saltstack/salt/pull/33180 fixes this issue.

[tmehlinger]

For what it's worth, I've managed to work around this with a (rather convoluted) set of states that guarantee the latest available pip is installed as the system pip. I _would not_ consider this safe to use in the general case but it's working for me on fresh systems:

apt_python_packages_are_absent:
  pkg.removed:
    - pkgs:
      - python-pip
      - python-wheel
      - python-virtualenv

system_pip_is_installed:
  cmd.run:
    # did this the most convoluted way imaginable because `curl | python` wasn't
    # working for some reason
    - name: |
        DIR=$(mktemp -d)
        [[ -d $DIR ]] || exit 1
        pushd $DIR
        curl -qO https://bootstrap.pypa.io/get-pip.py
        /usr/bin/python get-pip.py
        popd
        rm -rf $DIR
    - unless: [[ -x $(which pip) ]]
    - require:
      - pkg: apt_python_packages_are_absent

system_pip_has_pyopenssl:
  pip.installed:
    - pkgs:
      - ndg-httpsclient
      - pyasn1
      - pyopenssl
    - require:
      - cmd: system_pip_is_installed

system_pip_is_recent:
  cmd.run:
    - name: /usr/bin/env pip install -U pip
    - require:
      - cmd: system_pip_is_installed

system_python_packages_are_installed:
  pip.installed:
    - name: virtualenv
    - require:
      - cmd: system_pip_is_recent

[doberloh]

+1 on a fix for this ASAP.

[thatch45]

Yes, for the record this hit us out of nowhere because of an upstream change and are working on this right now

[thatch45]

ok, can you try this patch out? It is not the most elegant, but should be safe

33383

[shawnbutts]

33383 did not work for me.

    Function: pip.installed
        Name: configparser
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.6/site-packages/salt/state.py", line 1624, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.6/site-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
                  self.gen.throw(type, value, traceback)
                File "/usr/lib/python2.6/site-packages/salt/utils/context.py", line 42, in func_globals_inject
                  yield
                File "/usr/lib/python2.6/site-packages/salt/loader.py", line 1492, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.6/site-packages/salt/states/pip_state.py", line 609, in installed
                  ret['comment'] = '\n'.join(comments)
              TypeError: sequence item 0: expected string, NoneType found

[thatch45]

Thanks @shawnbutts , can you try this again> I moved the try/execpt to the right spot:

33383

[shawnbutts]

That did it! Thanks for the quick work.

@tmehlinger can you confirm as well?

[tmehlinger]

Like a champ here. :)

@thatch45 are you referring to the upstream change in pip? Don't sweat it... it also caught me out with devpi. :(

[thatch45]

thanks @tmehlinger ! Glad this got things working again, and thanks for letting me vent a little!

[tmehlinger]

Sure thing man, nice running into you again, one way or another. :)

[thatch45]

it is, looks like life has been a ride since firehost for you!

[thatch45]

This is serious enough that I think we will cut a 2015.8.10, upstream apis changed at the last minute can really throw us for a loop :)

[cachedout]

This should be fully resolved, per @thatch45