Vagrant: Unexpected ansible-playbook command line

[promethe42]

Vagrant version

$ vagrant -v
Vagrant 2.2.9

Host operating system

Ubuntu 20.04

Guest operating system

Ubuntu 18.04

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.require_version ">= 2.2.9"

Vagrant.configure("2") do |config|
  config.vagrant.plugins = {
    "vagrant-env" => { "version" => "0.0.3" },
    "vagrant-disksize" => { "version" => "0.1.3" },
  }

  config.vm.define "git" do |host|
    host.vm.box = "ubuntu/bionic64"
    host.vm.hostname = "git.aerys.in.test"
    host.vm.network :private_network, ip: "192.168.42.2"

    host.env.enable

    host.vm.provider "virtualbox" do |v|
      v.memory = 4096
      v.cpus = 2
    end

    host.vm.provision "git", type: "ansible_local" do |ansible|
      ansible.version = "2.8.0"
      ansible.install_mode = "pip"
      ansible.playbook = "provisioning/playbook.yml"
      ansible.verbose = 'v'
      ansible.inventory_path = "provisioning/hosts.yml"
      ansible.become = true
      ansible.skip_tags = ["dcl2gitlab"],
      ansible.extra_vars = {
        ansible_ssh_user: "vagrant",
      }
    end
end

Debug output

vagrant_provision_git.txt

Expected behavior

The ansible-playbook command line should be:

cd /vagrant && PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ansible-playbook --connection="local" --limit="git" --inventory-file=provisioning/hosts.yml --extra-vars=\{\"base_hostname\":\"aerys.in.test\",\"ansible_ssh_user\":\"vagrant\"\} --become -v --skip-tags=dcl2gitlab provisioning/playbook.yml

Actual behavior

$ vagrant provision git
==> git: Running provisioner: git (ansible_local)...
    git: Running ansible-playbook...
cd /vagrant && PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ansible-playbook --limit="git" --inventory-file=provisioning/hosts.yml --extra-vars=\{\"base_hostname\":\"aerys.in.test\",\"ansible_ssh_user\":\"vagrant\"\} --become -v --skip-tags=dcl2gitlab,{:base_hostname=>"aerys.in.test", :ansible_ssh_user=>"vagrant"} provisioning/playbook.yml
ERROR! the playbook: :ansible_ssh_user= could not be found
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Please note how:

• --skip-tags=dcl2gitlab is not concatenated to anything else
• --connection="local" has been added

Steps to reproduce

1. vagrant up git with the provided Vagrantfile

References

None AFAIK.

[promethe42]

If I remove:

      ansible.skip_tags = ["dcl2gitlab"],

then the command line is valid:

==> git: Running provisioner: git (ansible_local)...
    git: Running ansible-playbook...
cd /vagrant && PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ansible-playbook --limit="git" --inventory-file=provisioning/hosts.yml --extra-vars=\{\"base_hostname\":\"aerys.in.test\",\"ansible_ssh_user\":\"vagrant\"\} --become -v provisioning/playbook.yml
Using /etc/ansible/ansible.cfg as config file
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to 
allow bad characters in group names by default, this will change, but still be 
user configurable on deprecation. This feature will be removed in version 2.10.
 Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
 [WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details


PLAY [server:&linux] ***********************************************************

TASK [Gathering Facts] *********************************************************
fatal: [git]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Host key verification failed.", "unreachable": true}

PLAY RECAP *********************************************************************
git                        : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   

but still doesn't work because Ansible won't connect. That's why --connection=local must be added in ansible.raw_arguments. Which is unexpected for the ansible_local provisioner.

[gildegoma]

Hi @promethe42. Thank you for asking... this doesn't seem to be any vagrant bug, but I'd be happy to help you to fix your settings.

A few remarks and questions about the provided information:

1. The line ansible.skip_tags = ["dcl2gitlab"], ends with a comma (,) which doesn't make sense, and apparently leads to pick the rest of the provisioner ruby-block and append it to the --skip-tags argument (see debug output: --skip-tags=dcl2gitlab,{:base_hostname=>"aerys.in.test", :ansible_ssh_user=>"vagrant"}). Please just try to add back ansible.skip_tags = ["dcl2gitlab"], it should work as good as when you removed this line (second comment).
2. I don't get why you declare a extra_vars = { ansible_ssh_user: "vagrant" } when using the ansible_local provisioner. Do you want to configure your git VM with Ansible, or configure other targets from the git machine?
3. could you please give the content of your provisioning/hosts.yml inventory (or even better a complete github repo with your example and a description of what you want to achieve.

Thank you very much!

[promethe42]

Oh boy... this is awkward... :blush:

The line ansible.skip_tags = ["dcl2gitlab"], ends with a comma (,) which doesn't make sense, and apparently leads to pick the rest of the provisioner ruby-block and append it to the --skip-tags argument (see debug output: --skip-tags=dcl2gitlab,{:base_hostname=>"aerys.in.test", :ansible_ssh_user=>"vagrant"}). Please just try to add back ansible.skip_tags = ["dcl2gitlab"], it should work as good as when you removed this line (second comment).

Totally got caught off guard by the trailing-coma-so-I-don't-get-git-diff-for-nothing habit.

I really never thought this would be valid Ruby syntax... That is indeed the problem.

I don't get why you declare a extra_vars = { ansible_ssh_user: "vagrant" } when using the ansible_local provisioner. Do you want to configure your git VM with Ansible, or configure other targets from the git machine?

Kinda didn't matter at the time since it didn't get to that point. I removed it in the meantime. It is indeed not required.

could you please give the content of your provisioning/hosts.yml inventory (or even better a complete github repo with your example and a description of what you want to achieve.

Here you go:

all:
  hosts:
    git:
      ansible_host: "git.{{ base_hostname }}"
      ansible_ssh_private_key_file: "{{ lookup('env', 'GIT_AERYS_IN_SSH_KEY') }}"
      ansible_ssh_user: aerys
      ansible_become: yes

I guess the --connection=local is required because the ansible_local provisioner does not expect ansible_host to be set in the host_vars. But that value does match with the Vagrant VM hostname though (git.aerys.in.test in this case...).

[promethe42]

I still expect ansible_local to run ansible-playbook --connection=local by default though. Any reason/idea why it doesn't?

[gildegoma]

Any reason/idea why it doesn't?

Yes, the reason is that ansible_local provisioner means that the Ansible controller machine is the Vagrant guest (instead of being the Vagrant host when using ansible provisioner). Therefore the connection mode is not configured via --connection argument, but expected to be managed via the Ansible inventory.

See this example where the VM guest is used as ansible controller to manage other nodes.

All that said, and thank to your Ansible inventory example, I understand that you want to run ansible playbooks against the git VM. In such _standard_ use case, I would strongly recommend you to switch to the auto-generated inventory, which should perfectly cover your needs.
Something like that should do the trick:

    host.vm.provision "git", type: "ansible_local" do |ansible|
      ansible.version = "2.8.0"
      ansible.install_mode = "pip"
      ansible.playbook = "provisioning/playbook.yml"
      ansible.verbose = 'v'
      # TO BE REMOVED: ansible.inventory_path = "provisioning/hosts.yml"
      ansible.become = true
      ansible.skip_tags = ["dcl2gitlab"]
    end

If you do need to ssh into the git VM as a different user (as ansible_ssh_user: aerys and ansible_ssh_private_key_file: ... in your provisioning/hosts.yml suggest), please verify following points:

• Why an ssh connection onto the git guest VM with the vagrant (default) user is "not enough" (in a Vagrant dev/test context)
• If you really have to run some (or all) ansible tasks as aerys, then use some become_user option where needed (either globally or on a per-tasks/per-role basis)

I hope it helps...

[promethe42]

Here's what I've done:

• I've removed all the Ansible-related provisioners of my VMs.
• I've added one more VM at the end called "ansible": this VM will install and run ansible-playbook
• I've enabled --natdnshostresolver1 so that this VM can connect to the other VMs using their hostname (my host has the proper DNS entries in /etc/hosts)
• I've added the ANSIBLE_EXTRA_ARGS and ANSIBLE_LIMIT to offer a bit of control on how ansible-playbook runs.
• I've put the Ansible install in a Bash script that can also be used by the CI.

  config.vm.define "ansible" do |host|
    host.vm.box = "ubuntu/bionic64"
    host.vm.synced_folder ".", "/vagrant", mount_options: ['dmode=775,fmode=700']

    host.vm.provider "virtualbox" do |vb|
      vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
    end

    host.env.enable

    host.vm.provision "shell", path: "./script/install_ansible.sh"

    host.vm.provision "ansible_local" do |ansible|
      ansible.version = "2.8.0"
      ansible.install = false
      ansible.verbose = 'v'
      ansible.limit = ENV['ANSIBLE_LIMIT'] || "all:!build-windows"
      ansible.raw_arguments = ENV['ANSIBLE_EXTRA_ARGS']
      ansible.config_file = "provisioning/ansible.cfg"
      ansible.playbook = "provisioning/playbook.yml"
      ansible.inventory_path = "provisioning/hosts.yml"
      ansible.become = true
      ansible.playbook_command = "/vagrant/script/ansible-playbook.sh"
      ansible.extra_vars = {
        base_hostname: "aerys.in.test",
        ansible_ssh_user: "vagrant",
      }
    end
  end

This way, I can provision by running vagrant provision ansible.

If I want to provision a single machine, I can run ANSIBLE_LIMIT=vpn vagrant provision ansible.

Since the ansible machine is the last one, all the VMs are created and finally provisioned when the ansible one gets created. But I can still start and provision only a single VM if I want.

Since Ansible is on a control machine, not on the guests, it does mimic how the CI+production environment work.

And of course, since Ansible now runs in the ansible VM, it works flawlessly no matter what the host OS is.

All in all I'm pretty happy with the final result.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.