Vagrant: CentOS 8: ansible_local provisioner looks for Python2 packages, although Python3 is used

[bellackn]

When I want to setup a centos/8 box using the ansible_local provisioner and Python3, the provisioning process stops with the error below.

Vagrant version

2.2.6

Host operating system

Windows 10

Guest operating system

CentOS 8

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

VAGRANTFILE_API_VERSION = "2"
HOSTNAME = "foo"

# ssh port
SSH_PORT = 22022

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.box = "centos/8"
  config.vm.hostname = HOSTNAME

  # define sync folders
  config.vm.synced_folder "./", "/home/vagrant/sample", create: false, type: "rsync", rsync__auto: true, rsync__args: ["--verbose", "--archive", "-z", "--copy-links"]

  # force auto update of VirtualBox guest additions
  config.vbguest.auto_update = true

  config.vm.provider "virtualbox" do |vb|
    # set vm name
    vb.name = HOSTNAME
  end

  # provision controlhost
  config.vm.provision "ansible_local" do |ansible|
    ansible.provisioning_path = "/home/vagrant/sample/"
    ansible.install_mode = "pip"
    ansible.version = "2.7.13"
    ansible.compatibility_mode = "2.0"
    ansible.extra_vars = { ansible_python_interpreter:"/usr/bin/python3" }
    # playbook settings
    ansible.inventory_path = "inventory"
    ansible.playbook = "playbook.yml"
    ansible.limit = "all"
  end

end

Expected behavior

The ansible_local provisioner successfully sets up the box.

Actual behavior

$ vagrant provision
==> default: Configuring proxy environment variables...
==> default: Configuring proxy for Yum...
==> default: Running provisioner: ansible_local...
    default: Installing Ansible...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

dnf -y install curl gcc libffi-devel openssl-devel python-crypto python-devel python-setuptools

Stdout from the command:

Last metadata expiration check: 0:11:50 ago on Tue 07 Jan 2020 02:07:13 PM UTC.
Package curl-7.61.1-8.el8.x86_64 is already installed.
Package gcc-8.2.1-3.5.el8.x86_64 is already installed.
No match for argument: python-crypto
No match for argument: python-devel
No match for argument: python-setuptools


Stderr from the command:

Error: Unable to find a match

Steps to reproduce

1. Clone this sample repo.
2. Run vagrant up from there.

The lines in question seem to be these:
https://github.com/hashicorp/vagrant/blob/58687e6c4485790670fead7b579e3a1a60927b77/plugins/provisioners/ansible/cap/guest/redhat/ansible_install.rb#L36-L41
I guess one should check for the wanted Python version and use the appropriate packages then: python3-cryptography, python3-devel and python3-setuptools.

[bellackn]

For testing, I replaced the suggested package names locally on my machine and provisioning works fine now. I'd suggest that one should be able to set a parameter like ansible.pip_version = "3" to tell Vagrant to use pip3 and then switch the mentioned package names.
As a side effect, I could avoid this ugly hack to use pip3 from here:

    ansible.pip_install_cmd = "sudo dnf install -y python3-pip && sudo ln -s -f /usr/bin/pip3 /usr/bin/pip"

Otherwise, it will tell me pip: command not found.

[jbonhag]

Hi there,

Thanks for opening this issue (and especially for providing a repo, wow!). Another workaround would be to set ansible.install = false and install Ansible with a shell provisioner before the ansible_local provisioner in your Vagrantfile:

  config.vm.provision "shell", inline: "dnf install -y python3-pip && pip3 install --upgrade ansible==2.7.13"

With Python 2 reaching EOL, we should probably be using the Python 3 packages by default for pip installs anyways. We could check the package list for python3 variants and install those if they exist.

@gildegoma is this something you would be willing to look at?

[bellackn]

Hi @jbonhag, thanks for your reply. :) Of course, installing Ansible by a Shell script is a workaround, but I liked the way how I could neatly define all my parameters in the Vagrantfile. Also, as you already said, Python2 reached EOL and one should probably at least offer a convenient way to use Python3/pip3 here.

[gildegoma]

@bellackn thank you for your detailed problem report :heart:. Indeed CentOS 8 package naming is not backward compatible...

@jbonhag I've quickly looked at this, and here are the first points I could identify:

• Install new packages when CentoOS major version is >= 8 (python3-cryptography python3-devel python3-setuptools)
• At the moment, the default (and insecure :wink:) pip_install_cmd is not based on default package system but relies on https://bootstrap.pypa.io/get-pip.py script, and actually ends up installing pip3 as /usr/local/bin/pip (among others). The problem is that on the CentOS8 vagrant box, /usr/local/bin is not part of root's PATH.
• We probably could configure the provisioner to ensure (e.g. via alternatives commands) that python and pip "binaries" are in root's PATH and refer to the required versions. This way, it would also help to get rid of the ansible_python_interpreter: "/usr/bin/python3" extra_vars trick

As an example (just for testing, but not ready, nor final), here is a patched version of
plugins/provisioners/ansible/cap/guest/redhat/ansible_install.rb which works with centos/8 guest:

            def self.pip_setup(machine, pip_install_cmd = "")
              rpm_package_manager = Facts::rpm_package_manager(machine)

              machine.communicate.sudo("#{rpm_package_manager} -y install curl gcc libffi-devel openssl-devel python3-cryptography python3-devel python3-setuptools")
              machine.communicate.sudo("alternatives --set python /usr/bin/python3 ")

              Pip::get_pip machine, pip_install_cmd
              machine.communicate.sudo("alternatives --install /usr/bin/pip pip /usr/local/bin/pip 1")
            end

I assigned myself, as I'll try to work on a proper fix (which will be a good occasion to bring the missing unit tests for RHEL family, see #11227 and #6633). But any help is more than welcome (especially if we want to bring this into the next release) as I may lack of time in the future weeks. Please announce you here if some folk want to work on a fix.

[ymmmtym]

Hi there.

My environment is following versions.

• Mac OS X == 10.15.6
• vagrant == 2.2.9
• virtualbox == 6.1.8r137981

and I don't need vbguest, so I try without config.vbguest.auto_update = true

In my case, I would resolve python version problem using ansible.install_mode = "pip3".

But required ansible version is newer than 2.9.13 currently. Otherwise..

$ vagrant up
==> default: Running provisioner: ansible_local...
    default: Installing Ansible...
The requested Ansible version (2.7.13) was not found on the guest.
Please check the Ansible installation on your Vagrant guest system (currently: 2.9.13),
or adapt the provisioner `version` option in your Vagrantfile.
See https://docs.vagrantup.com/v2/provisioning/ansible_common.html#version
for more information.

Accordingly, I fix Vagrantfile like following.

# -*- mode: ruby -*-
# vi: set ft=ruby :

VAGRANTFILE_API_VERSION = "2"
HOSTNAME = "foo"

# ssh port
SSH_PORT = 22022

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.box = "centos/8"
  config.vm.hostname = HOSTNAME

  # define sync folders
  config.vm.synced_folder "./", "/home/vagrant/sample", create: false, type: "rsync", rsync__auto: true, rsync__args: ["--verbose", "--archive", "-z", "--copy-links"]

  # force auto update of VirtualBox guest additions
  # config.vbguest.auto_update = true # fixed for my environment

  config.vm.provider "virtualbox" do |vb|
    # set vm name
    vb.name = HOSTNAME
  end

  # provision controlhost
  config.vm.provision "ansible_local" do |ansible|
    ansible.provisioning_path = "/home/vagrant/sample/"
    ansible.install_mode = "pip3" # fixed
    ansible.version = "2.9.13" # fixed
    ansible.compatibility_mode = "2.0"
    ansible.extra_vars = { ansible_python_interpreter:"/usr/bin/python3" }
    # playbook settings
    ansible.inventory_path = "inventory"
    ansible.playbook = "playbook.yml"
    ansible.limit = "all"
  end

end

Then, I try vagrant ssh and show python versions.

[vagrant@foo ~]$ python3
python3     python3.6   python3.6m
[vagrant@foo ~]$ which python3
/usr/bin/python3
[vagrant@foo ~]$ pip
pip-3    pip3     pip-3.6  pip3.6
[vagrant@foo ~]$ which pip3
/usr/bin/pip3
[vagrant@foo ~]$ pip3 freeze
ansible==2.9.11
asn1crypto==0.24.0
Babel==2.5.1
bcrypt==3.1.6
cffi==1.11.5
configobj==5.0.6
cryptography==2.3
decorator==4.2.1
gpg==1.10.0
idna==2.5
iniparse==0.4
Jinja2==2.10.1
jmespath==0.9.0
MarkupSafe==0.23
netifaces==0.10.6
paramiko==2.4.3
pciutils==2.3.6
perf==0.1
ply==3.9
pyasn1==0.3.7
pycparser==2.14
pygobject==3.28.3
PyNaCl==1.3.0
pyOpenSSL==18.0.0
python-dateutil==2.6.1
python-dmidecode==3.12.2
python-linux-procfs==0.6
pytz==2017.2
pyudev==0.21.0
PyYAML==3.12
rhnlib==2.8.6
rpm==4.14.2
schedutils==0.6
six==1.11.0
slip==0.6.4
slip.dbus==0.6.4
syspurpose==1.23.8

[ymmmtym]

If you want to manage ansible version via requirements.txt,
add following contexts to Vagrantfile.

ansible.install_mode = "pip3_args_only"
ansible.pip_args = "-r requirements.txt"

[gildegoma]

@ymmmtym there are no install_mode options like "pip3" or "pip3_args_only" (but "pip" and "pip_args_only" exist).

[elreydetoda]

@ymmmtym there are no install_mode options like "pip3" or "pip3_args_only" (but "pip" and "pip_args_only" exist).

It doesn't list it, but it just worked for me: https://github.com/elreydetoda/vagrant-files/blob/d294ed1fc4cbe709a6b342bb914f569ce56ad824/elrey741_kali-linux_amd64/Vagrantfile#L90