Salt: Missing m2crypto dependency breaks x509 module

[carlpett]

Salt 2015.5 had a dependency on the m2crypto package (at least the rpms), which seems to be removed in 2015.8 and 2016.3. I just installed our first minions which went directly onto 2016.3 without having 2015.5 at some point, and therefore I didn't get m2crypto installed. This breaks the x509 module, since it requires m2crypto: 'x509' __virtual__ returned False: Could not load x509 state: m2crypto unavailable

Since installing all potential dependencies would be quite heavy and undesireable, I'm just reporting this because of the regression.

Setup

Centos 7.2, salt-minion 2016.3.1

Versions Report

Salt Version:
           Salt: 2016.3.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed <--
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.22.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

[Ch3LL]

I believe we moved away from m2crypto as a default dependency since we use pycrypto now. @dmurphy18 can you confirm this?

If this is not a default dependency as a believe is the case then we should probably add to the x509 docs that it requires installing m2crypto as an additional step.

[dmurphy18]

@carlpett The dependency on m2crypto was removed with 2015.8.0 and since that release only python-crypto v2.6.1 has been required. For the moment you could install m2crypto specifically from the recent release 2015.8.11 on repo.saltstack.com for your platform.

I shall look into the issue next week as to what can be done for x509.

[Ch3LL]

For now i'll lable this a documentation bug since we could add m2crypto as a dependency on x509 module docs.

[haraldrudell]

For Ubuntu 16.10:

apt-get install python-m2crypto

[oliver-dungey]

This still seems to be a problem with 2017.7.1 on CentOS 6.9 but with additional issues - If I do a Salt versions report it wrongly reports that M2Crypto is not installed:

[root@aldvmnftsm01 salt]# salt --versions-report
Salt Version:
           Salt: 2017.7.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed <----- WRONG!
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Jul 12 2017, 17:32:34)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 6.9 Final
         locale: UTF-8
        machine: x86_64
        release: 2.6.32-696.10.3.el6.x86_64
         system: Linux
        version: CentOS 6.9 Final

I do have m2crypto installed:

[root@blah salt]# yum list installed | grep -iu 'm2crypto'
m2crypto.x86_64        0.20.2-9.el6     @base

And this means that when I try and use the x509 state module I get this:

Comment: State 'x509.private_key_managed' was not found in SLS 'blah.blah-configured'
              Reason: 'x509' __virtual__ returned False: Could not load x509 state: m2crypto unavailable

So I wondered if m2crypto != M2Crypto and tried pip install:

[root@aldvmnftsm01 salt]# pip install M2Crypto
/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
You are using pip version 7.1.0, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Requirement already satisfied (use --upgrade to upgrade): M2Crypto in /usr/lib64/python2.6/site-packages

Then I looked back at the Salt versions report above and noticed the mix of 2.6, 2.7 and 2.8 across Python and Jinja modules - am I in Python version hell? Looks to me like Salt has it's own Python 2.7 yet the CentOS default is 2.6, no idea how to get around this. Any help much appreciated.

[oliver-dungey]

I've found my problem and a workaround: on CentOS 6.9 using Salt 2017.7.1 you end up with Python 2.6 and 2.7 alongside each other. This makes things a bit tricky so I had to do the following to get x509 to work:

yum install -y python27-pip python27-devel gcc openssl-devel
pip27 install M2Crypto

After running this I get the right results in the versions-report:

Salt Version:
           Salt: 2017.7.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.26.4
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Jul 12 2017, 17:32:34)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 6.9 Final
         locale: UTF-8
        machine: x86_64
        release: 2.6.32-696.10.3.el6.x86_64
         system: Linux
        version: CentOS 6.9 Final

[oliver-dungey]

I've just updated to 2017.7.4 and there is still a dependency on M2Crypto - the X509 module still needs that dependency.

[Ch3LL]

as you can see here: https://github.com/saltstack/salt-pack/issues/481 we are readding m2crypto into our packaging in oxygen :)

[rnickle]

I'm having problems with this on Ubuntu 18.04 Bionic, which points to the difficulty in getting M2Crypto working with Python3. Has anybody had any luck with that?

utility@ubbt3dt1:~$ salt-minion --versions-report
Salt Version:
           Salt: 2018.3.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.5 (default, Apr  1 2018, 05:46:30)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-24-generic
         system: Linux
        version: Ubuntu 18.04 bionic

          ID: Docker_TLS_CA_Cert
    Function: x509.pem_managed
        Name: /etc/ssl/certs/ca_docker-template.crt
      Result: False
     Comment: State 'x509.pem_managed' was not found in SLS 'Docker.tls'
              Reason: 'x509' __virtual__ returned False: Could not load x509 state: m2crypto unavailable

[eliasp]

I'm having problems with this on Ubuntu 18.04 Bionic, which points to the difficulty in getting M2Crypto working with Python3. Has anybody had any luck with that?

Same for me!
Is there a (technical) reason (besides noone did it so far) for not porting x509 to pycrypto?

[dmurphy18]

@eliasp @rnickle There were problems in getting M2Crypto to build on Python 3 hence it is not in the list of packages for Salt in Python 3, see open-source url: https://github.com/saltstack/salt-pack-py3 which is used to build Salt for Python 3. Salt currently makes use pf pycrypto for Python 3.

Now that the latest point release is out of the way, hope to have another crack at getting it working: however would gladly accept a PR for working version, given there are a few irons in the fire.

[rnickle]

[UPDATE 20181119]

The m2crypto bug seems to be fixed in the release I'm running: 2018.3.3

To work with Salt Mine which uses module.run I did have to use this option:

use_superseded:

• module.run

But that is unrelated to this case.

I also am having some trouble getting my Mine to work, but that is also unrelated.

Thanks,

Rick

[ORIGINAL TEXT FOLLOW]

I ran into this bug, received a workaround which did work, but did not go
back and try the whole stack again, I hope to do so soon.

Thanks,

Rick

On Mon, Oct 29, 2018 at 7:27 PM David Murphy [e-mail]
wrote:

There were problems in getting M2Crypto to build on Python 3 hence it is
not in the list of packages for Salt in Python 3, see open-source url:
https://github.com/saltstack/salt-pack-py3 which is used to build Salt
for Python 3. Salt currently makes use pf pycrypto for Python 3.

Now that the latest point release is out of the way, hope to have another
crack at getting it working: however would gladly accept a PR for working
version, given there are a few irons in the fire.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/35013#issuecomment-434117752,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AfO5Al5TUtmDE5OhTrcn2n3h5Tvar1Bpks5up47tgaJpZM4JXC5c
.