Salt: docker-py requirement failing satisfaction

[warden]

Calling state.highstate yelds on every run:

----------
          ID: common_packages
    Function: pip.installed
        Name: docker-py >=1.4.0
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
     Started: 16:37:02.502184
    Duration: 58.691 ms
     Changes:
----------

even though the sts is like this:

common_packages:
  pkg.installed:
    - names:
      - htop
      - python-pip
  pip.installed:
    - name: docker-py >=1.4.0
    - require:
      - pkg: python-pip
    - reload_modules: True

(Masterless setup)

Versions:

root@trn:~# salt --versions
Salt Version:
           Salt: 2015.8.1

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
           Mako: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.0.1
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-65-generic
         system: Ubuntu 14.04 trusty

root@trn:~# pip list  | grep docker
docker-py (1.5.0)

[rallytime]

Hi @warden - Do you happen to have both pip and pip2 installed? I was hitting this the other day and I am still not sure what is happening.

[warden]

Interestingly, I do...

root@trn:~# md5sum /usr/bin/pip2 /usr/bin/pip
1bb4536af835f454fce147a8fe8fe7b2  /usr/bin/pip2
75ed212e969fe3edf4f81cf9518e2869  /usr/bin/pip
root@trn:~# pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
root@trn:~# pip2 --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)

root@trn:~# dpkg -S /usr/bin/pip
python-pip: /usr/bin/pip
root@trn:~# dpkg -S /usr/bin/pip2
python-pip: /usr/bin/pip2

[rallytime]

Ok good to know. Another interesting point I just discovered is that this is only reproducible when using masterless. I can reproduce this with salt-call state.sls pip, but not if I just use salt my-minion state.sls pip. Even though the states were run on the same minion. (My pip state is the same as yours, pretty much.)

[warden]

That's interesting. If I remember right, this STS works fine for the first time, only later it starts to report problems (after upgrading pip I guess?). Maybe some cache is still not invalidated?

[rallytime]

Another weird thing...If I have a local minion and use salt local-minion state.sls pip I also get the import errors...

[rallytime]

If I run the state multiple times from master to non-local minion, everything works fine.

First run from Master to Non-Local Minion:

# salt rally-u1 state.sls pip
rally-u1:
----------
          ID: pip_pkgs
    Function: pkg.installed
        Name: python-pip
      Result: True
     Comment: Package python-pip is already installed
     Started: 17:06:22.940420
    Duration: 795.113 ms
     Changes:
----------
          ID: pip_pkgs
    Function: pip.installed
      Result: True
     Comment: Python package docker-py was already installed
              All packages were successfully installed
     Started: 17:06:23.936146
    Duration: 6107.404 ms
     Changes:
              ----------
              apache-libcloud==0.18.0:
                  Installed
              azure==1.0.2:
                  Installed

Summary for rally-u1
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:   6.903 s

Second Run:

# salt rally-u1 state.sls pip
rally-u1:
----------
          ID: pip_pkgs
    Function: pkg.installed
        Name: python-pip
      Result: True
     Comment: Package python-pip is already installed
     Started: 17:17:00.449280
    Duration: 832.211 ms
     Changes:
----------
          ID: pip_pkgs
    Function: pip.installed
      Result: True
     Comment: Python package docker-py was already installed
              Python package azure was already installed
              Python package apache-libcloud was already installed
              All packages were successfully installed
     Started: 17:17:01.504502
    Duration: 1660.498 ms
     Changes:

Summary for rally-u1
------------
Succeeded: 2
Failed:    0
------------
Total states run:     2
Total run time:   2.493 s

Salt-Call on same Minion where the above two statements were successful, but salt-call fails:

# salt-call state.sls pip
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache u'salt://pip.sls'
[INFO    ] Running state [python-pip] at time 17:07:38.674105
[INFO    ] Executing state pkg.installed for python-pip
[INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO    ] Package python-pip is already installed
[INFO    ] Completed state [python-pip] at time 17:07:39.446300
[INFO    ] Running state [pip_pkgs] at time 17:07:39.548465
[INFO    ] Executing state pip.installed for pip_pkgs
[ERROR   ] An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
[INFO    ] Completed state [pip_pkgs] at time 17:07:39.549588
local:
----------
          ID: pip_pkgs
    Function: pkg.installed
        Name: python-pip
      Result: True
     Comment: Package python-pip is already installed
     Started: 17:07:38.674105
    Duration: 772.195 ms
     Changes:
----------
          ID: pip_pkgs
    Function: pip.installed
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
              An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
              An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
     Started: 17:07:39.548465
    Duration: 1.123 ms
     Changes:

Summary for local
------------
Succeeded: 1
Failed:    1
------------
Total states run:     2
Total run time: 773.318 ms

The same error happens if doing salt-call on the master and salt on a local minion.

Also I am testing these all at recent installs of the 2015.8 branch.

[rallytime]

@warden This might be a pip packaging problem. I was able to restore expected functionality with salt-call by doing:

apt-get remove python-pip
easy_install pip

I am still looking around, but it seems that the pip version 1.5.4 has trouble importing with:

ImportError: cannot import name IncompleteRead

I'm still looking around, but that might get you going for now.

[warden]

Good catch. So I tried it this way:

python-pip:
  pkg.removed
python-pip-whl:
  pkg.removed

pip-install:
  cmd.run:
    - name: easy_install pip

dockerpy:
  pip.installed:
    - name: docker-py >=1.4.0
    - require:
      - pkg: python-pip
      - pkg: python-pip-whl
      - cmd: pip-install
    - reload_modules: True

This works, but notice the python-pip-whl. Even with this scenario this was not working until I removed that package.

[dlanderson]

I just hit this as well. Removing python-pip, python-pip-whl and using easy_install worked for me

[rallytime]

Since this is a bug with pip 1.5.4 and the fix is to simply upgrade your version of pip see example state above, I am going to close this.

More information about this bug can be found in the following links:

• https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1306991
• http://stackoverflow.com/questions/27341064/how-do-i-fix-importerror-cannot-import-name-incompleteread/27341847#27341847
• http://askubuntu.com/questions/561377/pip-wont-run-throws-errors-instead

[gravyboat]

For anyone else that runs into this, the issue isn't resolved yet from the main Ubuntu repo.

[dragon788]

One thing I've done is to provide a cwd and bin_env for anything I need to call using pip, which allows me to tell it to specifically use /usr/bin/pip and not just import something called pip* that might show up in the class path.

[PKiman]

I got a similar error message when trying to install virtualenv from pip code base, but figured it out.

sudo salt 'minion-1' state.highstate test=False -l debug
      ID: virtualenv
Function: pip.installed
  Result: False
 Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
 Started: 00:38:49.330511
Duration: 0.525 ms
 Changes:   

Debugging the library:
/usr/lib/python2.6/site-packages/salt/states/pip_state.py

try:
    import pip
    HAS_PIP = True
except ImportError:
    HAS_PIP = False

...
    if not HAS_PIP:
        ret['comment'] = (
            'An importable pip module is required but could not be found on '
            'your system. This usually means that the system''s pip package '
            'is not installed properly.'
        )

In my case salt-master and salt-minion depends on python2.6, but pip depended on python2.7
If salt tries to "import pip" then an exception with the message is thrown, because pip not available in python 2.6.

ps aux|grep salt-minion
root      2616  0.0  3.1 742148 64396 ?        Sl   Jan10  20:25 /usr/bin/python2.6 /usr/bin/salt-minion -d

pip --version
pip 6.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)

Salt Version:
           Salt: 2015.8.1

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
           Mako: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.5.0
         Python: 2.6.9 (unknown, Dec 17 2015, 01:08:55)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: 0.9.0
        timelib: Not Installed

System Versions:
           dist:   
        machine: x86_64
        release: 4.1.13-18.26.amzn1.x86_64

The solution was remove the package: python27-pip and install python26-pip instead.

[gposton]

It took me far too long to find this @PKiman THANK YOU!