Salt: zeromq 4.1.4-5.el7 from EPEL causes issues with python-zmq-14.7.0-1.el7.x86_64 from saltstack repo

Created on 10 Jul 2016 · 25Comments · Source: saltstack/salt

Description of Issue/Question

Setup

CentOS 7 + EPEL repo
Saltstack repo enabled and salt* related packages installed

Steps to Reproduce Issue

Try installing the latest zeromq from EPEL, that came out earlier today and you'll get errors like so:

Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package zeromq.x86_64 0:4.0.5-4.el7 will be updated
--> Processing Dependency: libzmq.so.4()(64bit) for package: python-zmq-14.7.0-1.el7.x86_64
---> Package zeromq.x86_64 0:4.1.4-5.el7 will be an update
--> Finished Dependency Resolution
Error: Package: python-zmq-14.7.0-1.el7.x86_64 (@saltstack)
           Requires: libzmq.so.4()(64bit)
           Removing: zeromq-4.0.5-4.el7.x86_64 (@saltstack)
               libzmq.so.4()(64bit)
           Updated By: zeromq-4.1.4-5.el7.x86_64 (epel)
              ~libzmq.so.5()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Versions Report

Salt Version:
           Salt: 2015.8.10

Dependency Versions:
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
           Mako: 0.8.1
         PyYAML: 3.11
          PyZMQ: 14.7.0
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: 0.8.6
       cherrypy: 3.2.2
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: 1.4.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: 1.2.3
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 4.4.14.bs.ufd
         system: CentOS Linux 7.2.1511 Core

I get the same results with 2016.3.1 as it uses the same python-zmq.

Could you update/rebuilt python-zmq against the latest zeromq and provide that in your repos or if not, make the epoch on the one provided from saltstack higher than 0? For now, I'm excluding zeromq from EPEL to avoid issues.

Bug P1 Packaging Platform TEAM Platform fixed-pending-your-verification severity-medium

Source

seanjnkns

👍8

Most helpful comment

The fix for this shall appear in the next point release, which is soon.

dmurphy18 on 26 Jul 2016

🎉4

All 25 comments

Same issue

$ salt --versions-report
Salt Version:
           Salt: 2016.3.1

Dependency Versions:
           cffi: 0.8.6
       cherrypy: 3.2.2
       dateutil: 2.5.3
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.3
        libgit2: 0.21.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: 0.21.4
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.18.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

timwsuqld on 11 Jul 2016

👍1

ping @dmurphy18 would you mind looking at this? Is this something we could implement. Thanks

Ch3LL on 11 Jul 2016

@seanjnkns @Ch3LL @timwsuqld I shall take a look into this

dmurphy18 on 11 Jul 2016

awesome thanks

Ch3LL on 11 Jul 2016

I am changing the labels here as I think this is rather a bug than a feature, especially based on the additional report filed by #34629 (which I closed as a duplicate).

rallytime on 13 Jul 2016

We are affected by this too. Currently we work around the problem by excluding Zeromq packages from EPEL.

sevenval-admins on 14 Jul 2016

Same thing here on RHEL7

[root@el7repo bkeep]# yum update
Loaded plugins: product-id, search-disabled-repos, subscription-manager
epel/x86_64/metalink                                                                                                                             |  13 kB  00:00:00
epel                                                                                                                                             | 4.3 kB  00:00:00
rhel-7-server-eus-rpms                                                                                                                           | 3.7 kB  00:00:00
rhel-7-server-optional-rpms                                                                                                                      | 3.5 kB  00:00:00
rhel-7-server-rpms                                                                                                                               | 3.7 kB  00:00:00
rhel-ha-for-rhel-7-server-eus-rpms                                                                                                               | 3.1 kB  00:00:00
rhel-rs-for-rhel-7-server-eus-rpms                                                                                                               | 3.1 kB  00:00:00
salt-2016.3                                                                                                                                      | 2.9 kB  00:00:00
(1/2): epel/x86_64/updateinfo                                                                                                                    | 585 kB  00:00:00
(2/2): epel/x86_64/primary_db                                                                                                                    | 4.3 MB  00:00:01
Resolving Dependencies
--> Running transaction check
...
---> Package zeromq.x86_64 0:4.0.5-4.el7 will be updated
--> Processing Dependency: libzmq.so.4()(64bit) for package: python-zmq-14.7.0-1.el7.x86_64
---> Package zeromq.x86_64 0:4.1.4-5.el7 will be an update
--> Processing Dependency: libsodium.so.13()(64bit) for package: zeromq-4.1.4-5.el7.x86_64
--> Running transaction check
---> Package libsodium.x86_64 0:1.0.5-1.el7 will be installed
---> Package zeromq.x86_64 0:4.0.5-4.el7 will be updated
--> Processing Dependency: libzmq.so.4()(64bit) for package: python-zmq-14.7.0-1.el7.x86_64
--> Finished Dependency Resolution
Error: Package: python-zmq-14.7.0-1.el7.x86_64 (@salt-2016.3)
           Requires: libzmq.so.4()(64bit)
           Removing: zeromq-4.0.5-4.el7.x86_64 (@epel)
               libzmq.so.4()(64bit)
           Updated By: zeromq-4.1.4-5.el7.x86_64 (epel)
              ~libzmq.so.5()(64bit)
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************

Error: Package: python-zmq-14.7.0-1.el7.x86_64 (@salt-2016.3)
           Requires: libzmq.so.4()(64bit)
           Removing: zeromq-4.0.5-4.el7.x86_64 (@epel)
               libzmq.so.4()(64bit)
           Updated By: zeromq-4.1.4-5.el7.x86_64 (epel)
              ~libzmq.so.5()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

[root@el7repo bkeep]# salt-minion --versions
Salt Version:
           Salt: 2016.3.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.21.1
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.5 (default, Oct 11 2015, 17:47:16)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: redhat 7.2 Maipo
        machine: x86_64
        release: 3.10.0-327.22.2.el7.x86_64
         system: Linux
        version: Red Hat Enterprise Linux Server 7.2 Maipo

Same as above adding exclude=zeromq* to /etc/yum.repos.d/epel.repo takes care of the issue for the time being.

alias454 on 15 Jul 2016

FYI -- currently working a solution for this issue, no time frame for solution yet.

dmurphy18 on 15 Jul 2016

EPEL's zeromq has been updated to 4.1.4. Since you maintain the dependencies for RHEL systems without EPEL in the Salt repo you could update the Salt repo to the 4.1.4 from EPEL and rebuild python-zmq against the new zeromq.

If python-zmq won't compile against zeromq 4.1.4 typically what is done in RPM world is a "zeromq40" package would be created that has a slightly different prefix (you can use "obsoletes" to transition your package from zeromq to zeromq40 if that is the path you take) - see openssl098e or the gtk 2 vs 3 packages for examples.

As an engineer for a big enterprise I also suggest that you split up your repos into two - stuff you develop and dependencies your stuff needs. You are always going to have a challenge of coordinating updates to your repos with updates from EPEL and will run into this type of issue over and again. Some companies are going to prefer EPEL and not want to deal with the headache of different providers for the same packages while others are going to thank you for providing the dependencies because they can't get all of EPEL through their legal departments.

jbnance on 18 Jul 2016

Just FYI I'm currently not sure that I can exclude zeromq from EPEL7 as in our environment we have EPEL being sync'ed as a Red Hat Satellite Channel. All the packages for EPEL are provided by the RHN plugin inside of yum on our subscribed systems. Although I've not exhausted my resources (as I've fixed the small handful of machines suffering from this issue) I don't think it is likely I can perform any excludes on RHN channels.

EDIT: Adding "exclude=zeromq-4.1.4-5.el7.x86_64" to "/etc/yum.conf" will handle this particular problem when EPEL is a Red Hat Satellite Channel. Unfortunately I don't see any way of adding a "--exclude=zeromq-4.1.4-5.el7.x86_64" command-line argument to "yum" when performing a "pkg.upgrade" (for handling this work-around as a run-time option.)

PeterS242 on 18 Jul 2016

Having the same issue here. Salt 2016.3.1, CentOS 7.

danlsgiga on 26 Jul 2016

The fix for this shall appear in the next point release, which is soon.

dmurphy18 on 26 Jul 2016

🎉4

Confirmed resolved with the 2015.8.11 release. Once 2016.3.2 is released, this should be resolved completely and can be closed.

seanjnkns on 29 Jul 2016

@seanjnkns Thanks for the confirmation on 2015.8.11, 2016.3.2 is coming out soon

dmurphy18 on 29 Jul 2016

I can confirm that 2016.3.2 resolves this issue under CentOS 7. Updating from 2016.3.1 works fine, with no RPM dependency errors.

EugeneKay on 3 Aug 2016

🎉2

Hi,

I am getting this error when attempting to update

Resolving Dependencies
--> Running transaction check
---> Package python-zmq.x86_64 0:14.7.0-1.el7 will be updated
---> Package python-zmq.x86_64 0:15.3.0-2.el7 will be an update
--> Processing Dependency: libzmq.so.5()(64bit) for package: python-zmq-15.3.0-2.el7.x86_64
--> Finished Dependency Resolution
Error: Package: python-zmq-15.3.0-2.el7.x86_64 (NIWA_Saltstack_Saltstack-2015_8-x86_64)
Requires: libzmq.so.5()(64bit)

Dependency resolving failed due to missing dependencies.
Some repositories on your system are disabled, but yum can enable them
and search for missing dependencies. This will require downloading
metadata for disabled repositories and may take some time and traffic.

Enable all repositories and try again? [y/N]: y
NIWA_CentOS7_cloud_x86_64_openstack-kilo | 2.1 kB 00:00:00
NIWA_CentOS7_cloud_x86_64_openstack-kilo/updateinfo | 93 B 00:00:00
NIWA_CentOS7_cloud_x86_64_openstack-kilo/primary | 310 kB 00:00:00
NIWA_CentOS7_cloud_x86_64_openstack-kilo 1193/1193
--> Running transaction check
---> Package python-zmq.x86_64 0:15.3.0-2.el7 will be an update
--> Processing Dependency: libzmq.so.5()(64bit) for package: python-zmq-15.3.0-2.el7.x86_64
--> Finished Dependency Resolution
Error: Package: python-zmq-15.3.0-2.el7.x86_64 (NIWA_Saltstack_Saltstack-2015_8-x86_64)
Requires: libzmq.so.5()(64bit)
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

Any idea what's missing/happening!? Thanks :-)

dbaars on 26 Aug 2016

@dbaars The library libzmq.so.5 is from the package for zeromq-4.1.4-5.,el7.x86_64, available from both repo.saltstack.com and epel. Review the list of repo's that are enabled to ensure that the zeromq-4.1.4 package is available.

dmurphy18 on 26 Aug 2016

yeah it appears that salt minion needs to upgrade their zeromq library. this is causing issues still in centos.

openfly on 19 Jan 2017

This issue should be re-opened. It is a critical problem.

openfly on 19 Jan 2017

Re-opening to gather more information. I personally haven't seen this issue again and I'm running the following:

Salt Version:
Salt: 2016.3.4

Dependency Versions:
cffi: 0.8.6
cherrypy: 3.2.2
dateutil: 1.5
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: 1.4.3
M2Crypto: 0.21.1
Mako: 0.8.1
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: 1.2.3
pycparser: 2.14
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Sep 15 2016, 22:37:39)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4

System Versions:
dist: centos 7.2.1511 Core
machine: x86_64
release: 4.4.42-1.el7.centos.x86_64
system: Linux
version: CentOS Linux 7.2.1511 Core

zeromq-4.1.4-5.el7.x86_64

seanjnkns on 19 Jan 2017

found the issue
not an epel-release issue. someone has been running:

curl -L https://bootstrap.saltstack.com -o /tmp/install_salt.sh

on some of my boxes... and that install script apparently statically sets the version for repo.saltstack and loads it as a repo. Which is overriding the epel-release package. And leaving these nodes stuck on whatever version they got when they ran that script.

openfly on 19 Jan 2017

Here from you can get latest zeromq
wget https://copr.fedoraproject.org/coprs/saltstack/zeromq4/repo/epel-6/saltstack-zeromq4-epel-6.repo

mansurali901 on 10 May 2017

Is there a solution for people that are trying to bootstrap? I'm receiving this error while trying to bootstrap existing boxes using the salty cloud provider. version 2016.11.7