Salt: PSUtil 2.2 incompatible II

[guettli]

Description of Issue/Question

I guess this is a duplicate of #31748.

Steps to Reproduce Issue

salt:/srv # salt-ssh  'foo-minion' test.ping
Traceback (most recent call last):
  File "/usr/bin/salt-ssh", line 11, in <module>
    sys.exit(salt_ssh())
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 385, in salt_ssh
    import salt.cli.ssh
  File "/usr/lib/python2.7/site-packages/salt/cli/ssh.py", line 5, in <module>
    import salt.client.ssh
  File "/usr/lib/python2.7/site-packages/salt/client/__init__.py", line 30, in <module>
    import salt.minion
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 57, in <module>
    import salt.utils.psutil_compat as psutil
  File "/usr/lib/python2.7/site-packages/salt/utils/psutil_compat.py", line 16, in <module>
    import psutil
  File "/usr/lib64/python2.7/site-packages/psutil/__init__.py", line 89, in <module>
    import psutil._pslinux as _psplatform
AttributeError: 'module' object has no attribute '_pslinux'

Versions Report

pip freeze | grep -i psu
psutil==2.2.1

Version Report does not work, same exception like above :-(

If you import it in the interpreter, you get more info:

>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/psutil/__init__.py", line 181, in <module>
    raise ImportError(msg)
ImportError: version conflict:
    '/usr/lib64/python2.7/site-packages/psutil/_psutil_linux.so' 
    C extension module was built for another version of psutil (different than 2.2.1)
>>>

This one was installed via RPM

rpm -qf /usr/lib64/python2.7/site-packages/psutil/__init__.py
python-psutil-2.2.1-46.2.x86_64

After uninstalling the RPM psutil on the master, everything was fine.

I created this issue to close it, to make the solution easier to find for others.

[visbits]

This still isn't a fix. You can't just uninstall psutil to fix salt.. a lot of applications _all of openstack_ need it.

[guettli]

@visbits you said "You can't just uninstall psutil to fix salt.. a lot of applications all of openstack need it.".

I don't understand what you said. Salt does not depend on openstack. If openstack needs psutils, then openstack needs to care for this. I have solved psutils issues in salt like this, too: Just uninstall psutils and it works.

Please explain what you mean with "a lot of applications all of openstack need it.".

[visbits]

Uninstalling a python lib thats included standard with redhat to fix salts code? There's nothing about that which makes any sense what so ever... oh your engine in your car is knocking? well just remove it!!!

[zmingee]

Can confirm, I get the same issue. psutils is a _very, very_ common lib. My infrastructure uses OpenStack as well, and it specifically needs psutils. If I want to manage it with Salt, that presents an issue. Uninstalling a system wide package that is included with RHEL/CentOS is NOT A FIX.

If you cannot support the most recent version, that's in the official repositories of the linux distros, then your packaging team needs to be packaging the application inside of a virtualenv and distributing that. Because this psutils stuff bites me in the ass at least once a week.

[guettli]

I am sorry, I was wearing the "virtualenv glasses". Yes, uninstalling putils is not an option here.

Since salt should work without and with very new and very old version of psutil, I think one clean solution is to use import psutil at exactly one place in the salt code. In a new file called (for example) saltpsutil.py

Then provide all needed methods there and use import saltpsutil in the salt code.

This would keep the salt code clean. Up to now I see a lot of if HAS_PSUTIL:... in the source. It would be very nice if these ugly conditions could be handled in saltpsutil.py.

[jyfletcher]

Hi,

I found that two files in /usr/lib64/python2.7/site-packages/psutil:
_psutil_linux.so
_psutil_posix.so

Were not reinstalled after a remove and install of the python-psutil package on RHEL7. I don't know much about psutil but it looks to me like they are from an older version of the package and aren't removed during an upgrade... maybe psutil was changed to be pure python. As you can see in the "import psutil" error message above there is a version mismatch between the .so file and what the python lib is expecting. I think this supports the assumption.

So on another broken system I removed those two files manually and salt seems to be working fine without any other changes.

Oh and for me it was not that salt-ssh wouldn't work. For me, the salt-minion would not start. It had the same AttributeError error message.

[jyfletcher]

And yes, I see that this issue is closed, but I just ran across the problem and this thread. If there is somewhere else to put this info please let me know.

[visbits]

Thats fine until yum updates again and steamrollers over it.

---

Tyler Bishop
Founder EST 2007

O: 513-299-7108 x10
M: 513-646-5809
[ http://beyondhosting.net/ | http://BeyondHosting.net ]

This email is intended only for the recipient(s) above and/or otherwise authorized personnel. The information contained herein and attached is confidential and the property of Beyond Hosting. Any unauthorized copying, forwarding, printing, and/or disclosing any information related to this email is prohibited. If you received this message in error, please contact the sender and destroy all copies of this email and any attachment(s).

From: "Innectic" [e-mail]
To: "saltstack/salt" [e-mail]
Cc: "Tyler Bishop" [e-mail], "Mention" [e-mail]
Sent: Friday, November 10, 2017 5:57:47 PM
Subject: Re: [saltstack/salt] PSUtil 2.2 incompatible II (#35846)

Even though the issue is closed, thought I'd share what fixed it for me: pip install psutil --upgrade . This is running under Centos 7.

—
You are receiving this because you were mentioned.
Reply to this email directly, [ https://github.com/saltstack/salt/issues/35846#issuecomment-343608961 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AA1UGtZYviAdcYrcJ0DUxWaYDSM-vaORks5s1NTqgaJpZM4JvMje | mute the thread ] .

[zmingee]

_A_ fix for this might be running Salt in a virtualenv, but that would be counterintuitive since it's distrubuted as a package. Either Salt should use psutil in a backwards compatible way for the many users that have LTS distros, or Salt should just contain a wrapper library or fork psutil into the Salt repo if it's that big of a deal to modify the way it's used.

[arthurlogilab]

We have something similar happening on RHEL7, not sure why yet.

At some point we end up having :

>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/psutil/__init__.py", line 181, in <module>
    raise ImportError(msg)
ImportError: version conflict: '/usr/lib64/python2.7/site-packages/psutil/_psutil_linux.so' C extension module was built for another version of psutil (different than 2.2.1)

And removing /usr/lib64/python2.7/site-packages/psutil/_psutil_linux.so which doesn't belong to any package seem to solve it.

Am not sure if this is a new issue or not or if it is related to the new python-psutil which is available on some of RHEL servers :

# yum --showduplicates list python-psutil
    Available Packages
    python-psutil.x86_64 2.2.1-1.el7     salt-2017.7                                
    python-psutil.x86_64 2.2.1-1.el7     saltstack-pkgrepo                          
    python-psutil.x86_64 5.0.1-2.el7rhgs rhui-rh-gluster-3-web-admin-agent-for-rhel-7-server-rhui-rpms