Vagrant: Ansible: Both generated and static inventory

[ephracis]

It would be great if it was possible to use both a generated inventory (for hostnames) and a static inventory folder/file (for defining subgroups and hierarchy).

We are using Vagrant to generate dynamic labs to test our Ansible roles and plays. The hostnames used in the lab are dynamic and unique for each user and project combination. So we really need to use generated inventories for mapping hostnames to groups in the inventory.

At the same time, we also have complex group hierarchies in our Ansible inventories which are sometimes shared with other projects. Right now we need to specify these hierarchies multiple times in our Vagrantfile (as well as in ini-files which are used to deploy to production, Vagrant is only used during development).

I could implement this feature and send a PR but first I would like some discussion on how to best design such as feature and what it should look like in regard to user-facing options in the Ansible provisioner.

[ephracis]

One idea would be to make it possible to specify inventory_path as an array with the special keyword 'GENERATED' mean that we use a generated inventory. Like so:

Vagrant.configure("2") do |config|
  config.vm.provision "ansible" do |ansible|
    ansible.inventory_path = ["GENERATED", "ansible/hosts"]
  end
end

[gildegoma]

See also #7619

---

PS: I'll try to provide soon some more detailed answer (and tips).

[gildegoma]

use both a generated inventory (for hostnames) and a static inventory folder/file (for defining subgroups and hierarchy)

I see at least three possible ways to probably satisfy your use case(s), with current Vagrant capabilities, and will try to explain them below.

1) Use host_vars and group_vars directories beside the playbook file

From your question, I suppose that you want to place your host and group variables in files at the inventory level, but you also can locate them beside the playbook file.

So a typical setup would be:

VM1="vagrant-1"
Vagrant.configure("2") do |config|
  config.vm.box = "bento/centos-7"
  config.vm.provider "virtualbox" do |v|
    v.linked_clone = true
  end

  config.vm.define VM1 do |machine|
    machine.vm.hostname = VM1
  end

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.verbose = true
    ansible.groups = {
      web: [VM1],
      db: [VM1],
    }
  end

end

and a project directory structure like this:

├── ansible
│   ├── group_vars
│   │   ├── db
│   │   └── web
│   ├── host_vars
│   │   └── vagrant-1
│   └── playbook.yml
└── Vagrantfile

2) Control some Vagrant-specific variables from the Vagrantfile

If you need to override some variables for the Vagrant-specific guest machines, you can also get some nice DRY configuration by relying on the groups and host_vars option of the Ansible provisioner

As an example, the host_vars/vagrant-1 file could be removed and its variables set as in the following example:

...
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.groups = {
      web: [VM1],
      db: [VM1],
    }
    ansible.host_vars = {
      VM1 => { "nickname" => "gildegoma" },
    }
  end
...

3) Vagrant allows to explore unknown landscapes (a.k.a "unsupported" tricks)

:warning: Disclaimer the following tip is given as is, and is not considered as a safe way to use Vagrant tool. So "at your own risks"... :wink:

In order to allow a maximum of flexibility, the Ansible provisioner refers to the directory that contains the generated inventory. This way, it is possible on Unix/POSIX (or other symlink-friendly systems) to combine multiple inventories (managed by vagrant and not managed).

For example one could symlink an external inventory (or host_vars/group_vars directories) like illustrated in this Vagrantfile example:

Vagrant.configure("2") do |config|

  env = Vagrant::Environment.new()
  inventory_path = File.join(env.local_data_path, 'provisioners/ansible/inventory')
  link_target = File.join(env.cwd, '/etc/ansible/hosts')
  FileUtils::mkdir_p inventory_path
  FileUtils::ln_sf link_target, File.join(inventory_path, 'settled')

  config.vm.box = "bento/centos-7"
  config.vm.provider "virtualbox" do |v|
    v.linked_clone = true
  end

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
  end

end

[gildegoma]

@ephracis I'd really appreciate to get your feedback on the above, as I plan to use this as a basis to add more Tips/Tutorials to the Ansible Provisioner documentation in order to make these good practices more evident to new comers...
And of course, if your use cases wouldn't be covered, we keep talking... :+1:

[ephracis]

It's not so much about placement of variables (we use the group_vars folder for that). It is more the group-structure. Take a look at this for example:

[control]
node1.example.com

[network]
node1.example.com

[compute]
node2.example.com

[monitoring]
node1.example.com

[storage]
node2.example.com

[deployment]
localhost       ansible_connection=local

[baremetal:children]
control
network
compute
storage
monitoring

[chrony-server:children]
haproxy

[chrony:children]
control
network
compute
storage
monitoring

[collectd:children]
compute

[grafana:children]
monitoring

[etcd:children]
control
compute

[influxdb:children]
monitoring

[karbor:children]
control

[kibana:children]
control

[telegraf:children]
compute
control
monitoring
network
storage

[elasticsearch:children]
control

[haproxy:children]
network

[mariadb:children]
control

[rabbitmq:children]
control

[outward-rabbitmq:children]
control

[qdrouterd:children]
control

[mongodb:children]
control

[keystone:children]
control

[glance:children]
control

[nova:children]
control

[neutron:children]
network

[openvswitch:children]
network
compute
manila-share

[opendaylight:children]
network

[cinder:children]
control

[cloudkitty:children]
control

[freezer:children]
control

[memcached:children]
control

[horizon:children]
control

[swift:children]
control

[barbican:children]
control

[heat:children]
control

[murano:children]
control

[solum:children]
control

[ironic:children]
control

[ceph:children]
control

[magnum:children]
control

[sahara:children]
control

[mistral:children]
control

[manila:children]
control

[ceilometer:children]
control

[aodh:children]
control

[congress:children]
control

[panko:children]
control

[gnocchi:children]
control

[tacker:children]
control

[trove:children]
control

[tempest:children]
control

[senlin:children]
control

[vmtp:children]
control

[watcher:children]
control

[rally:children]
control

[searchlight:children]
control

[octavia:children]
control

[designate:children]
control

[placement:children]
control

[bifrost:children]
deployment

[zun:children]
control

[skydive:children]
monitoring

[redis:children]
control

[glance-api:children]
glance

[glance-registry:children]
glance

# Nova
[nova-api:children]
nova

[nova-conductor:children]
nova

[nova-consoleauth:children]
nova

[nova-novncproxy:children]
nova

[nova-scheduler:children]
nova

[nova-spicehtml5proxy:children]
nova

[nova-compute-ironic:children]
nova

[nova-serialproxy:children]
nova

[neutron-server:children]
control

[neutron-dhcp-agent:children]
neutron

[neutron-l3-agent:children]
neutron

[neutron-lbaas-agent:children]
neutron

[neutron-metadata-agent:children]
neutron

[neutron-vpnaas-agent:children]
neutron

[neutron-bgp-dragent:children]
neutron

[ceph-mon:children]
ceph

[ceph-rgw:children]
ceph

[ceph-osd:children]
storage

[cinder-api:children]
cinder

[cinder-backup:children]
storage

[cinder-scheduler:children]
cinder

[cinder-volume:children]
storage

[cloudkitty-api:children]
cloudkitty

[cloudkitty-processor:children]
cloudkitty

[freezer-api:children]
freezer

[iscsid:children]
compute
storage
ironic-conductor

[tgtd:children]
storage

# Karbor
[karbor-api:children]
karbor

[karbor-protection:children]
karbor

[karbor-operationengine:children]
karbor

# Manila
[manila-api:children]
manila

[manila-scheduler:children]
manila

[manila-share:children]
network

[manila-data:children]
manila

# Swift
[swift-proxy-server:children]
swift

[swift-account-server:children]
storage

[swift-container-server:children]
storage

[swift-object-server:children]
storage

# Barbican
[barbican-api:children]
barbican

[barbican-keystone-listener:children]
barbican

[barbican-worker:children]
barbican

# Heat
[heat-api:children]
heat

[heat-api-cfn:children]
heat

[heat-engine:children]
heat

# Murano
[murano-api:children]
murano

[murano-engine:children]
murano

# Ironic
[ironic-api:children]
ironic

[ironic-conductor:children]
ironic

[ironic-inspector:children]
ironic

[ironic-pxe:children]
ironic

# Magnum
[magnum-api:children]
magnum

[magnum-conductor:children]
magnum

# Sahara
[sahara-api:children]
sahara

[sahara-engine:children]
sahara

# Solum
[solum-api:children]
solum

[solum-worker:children]
solum

[solum-deployer:children]
solum

[solum-conductor:children]
solum

# Mistral
[mistral-api:children]
mistral

[mistral-executor:children]
mistral

[mistral-engine:children]
mistral

# Ceilometer
[ceilometer-api:children]
ceilometer

[ceilometer-central:children]
ceilometer

[ceilometer-notification:children]
ceilometer

[ceilometer-collector:children]
ceilometer

[ceilometer-compute:children]
compute

# Aodh
[aodh-api:children]
aodh

[aodh-evaluator:children]
aodh

[aodh-listener:children]
aodh

[aodh-notifier:children]
aodh

# Congress
[congress-api:children]
congress

[congress-datasource:children]
congress

[congress-policy-engine:children]
congress

# Panko
[panko-api:children]
panko

# Gnocchi
[gnocchi-api:children]
gnocchi

[gnocchi-statsd:children]
gnocchi

[gnocchi-metricd:children]
gnocchi

# Trove
[trove-api:children]
trove

[trove-conductor:children]
trove

[trove-taskmanager:children]
trove

# Multipathd
[multipathd:children]
compute

# Watcher
[watcher-api:children]
watcher

[watcher-engine:children]
watcher

[watcher-applier:children]
watcher

# Senlin
[senlin-api:children]
senlin

[senlin-engine:children]
senlin

# Searchlight
[searchlight-api:children]
searchlight

[searchlight-listener:children]
searchlight

# Octavia
[octavia-api:children]
octavia

[octavia-health-manager:children]
octavia

[octavia-housekeeping:children]
octavia

[octavia-worker:children]
octavia

# Designate
[designate-api:children]
designate

[designate-central:children]
designate

[designate-mdns:children]
network

[designate-worker:children]
designate

[designate-sink:children]
designate

[designate-backend-bind9:children]
designate

# Placement
[placement-api:children]
placement

# Zun
[zun-api:children]
zun

[zun-compute:children]
compute

# Skydive
[skydive-analyzer:children]
skydive

[skydive-agent:children]
compute
network

# Tacker
[tacker-server:children]
tacker

[tacker-conductor:children]
tacker

This file needs to be present when deploying to staging and production. It is really a pain to have to mimic this file in our Vagrantfile in order to use the same groups in the developer labs. So a nice solution would be to break this inventory file into two parts where one just defines the subgroups and the hierarchy. Then we could use that static file in Vagrantfile, and use the generated inventory file for placing the hostnames in the respective top-level groups.

Am I making any sense?

[ephracis]

So in short I would like to have an inventory like the one above, but without any hostnames, let's call it ansible/inventory_groups.

Then do something like this in my Vagrantfile:

...
# generate dynamic FQDNs to avoid collisions between developers/CI
...
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.inventory = ["__GENERATED__", "ansible/inventory_groups"]
    ansible.groups = {
      control: [dynamic_hostname_1],
      network: [dynamic_hostname_1],
      compute: [dynamic_hostname_2],
      monitoring: [dynamic_hostname_1],
      storage: [dynamic_hostname_2]
    }
  end
...

It looks like your third option will get me there. I was under the impression that Vagrant used only the file under .vagrant/provisioners/ansible/inventory and not the directory. if the directory is used, then it would absolutely be possible to do what I want without any modifications to the Vagrant code which is awesome.

I will try that one out and see if it works.

[ephracis]

Here is what I have done which seems to be working for my use case:

# Make inventory files available to Vagrant
#
# - `folder` - Folder where Ansible inventory files are stored, relative
#              the Vagrantfile
#
# Will link every file and folder inside the inventory-folder into Vagrants
# dynamic inventory. Files with the name `hosts` will be skipped to avoid
# linking in any static example hostnames.
def link_inventory_to_vagrant(folder)
  env = Vagrant::Environment.new
  static_inventory = File.join(env.cwd, folder)
  ansible_inventory = File.join(
    env.local_data_path,
    '/provisioners/ansible/inventory'
  )
  FileUtils.mkdir_p ansible_inventory
  Dir[File.join(static_inventory, '*')].each do |path|
    next if File.basename(path) == 'hosts' # skip linking hostnames
    FileUtils.ln_sf path, ansible_inventory
  end
end

Then all I need to do is put my inventory files into ansible/inventory and:

...
# generate dynamic FQDNs to avoid collisions between developers/CI
...
link_inventory_to_vagrant 'ansible/inventory'
...
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "ansible/playbook.yml"
    ansible.groups = {
      control: [dynamic_hostname_1],
      network: [dynamic_hostname_1],
      compute: [dynamic_hostname_2],
      monitoring: [dynamic_hostname_1],
      storage: [dynamic_hostname_2]
    }
  end
...

And now I can define all the subgroups and their hierarchy in a single location and use it both during deployment and my Vagrant labs.

[gildegoma]

@ephracis excellent! Yes you got it! I am keen to add such an example in the Tips and Tricks section of the Ansible Provisioner documentation. But since this pretty "advanced" usage, I don't want to modify Vagrant code to add more static/dynamic inventories behaviour.

[isakrubin]

@ephracis This seems to no longer work in vagrant 2.1.5 :/

`Vagrant failed to initialize at a very early stage:

There was an error loading a Vagrantfile. The file being loaded
and the error message are shown below. This is usually caused by
a syntax error.

Path: xxxxxxxxxxx
Line number: 0
Message: ThreadError: deadlock; recursive locking`

Commenting it out makes vagrant work fine again.

This works fine up to and including version 2.1.2

[amontalban]

@ephracis / @isakrubin I opened GH-10285 because I'm also seeing similar problem and I think it's related to env = Vagrant::Environment.new.

[chrisroberts]

@isakrubin This is due to the Environment instance attempting to read the Vagrantfile for loading local plugins, which causes it to recurse. You can prevent it by setting the vagrantfile_name option to an empty string:

env = Vagrant::Environment.new(vagrantfile_name: "")

[isakrubin]

@chrisroberts Setting it to empty string gets rid of the deadlock issue but results in env.local_data_path being nil which kinds of just moves the issue. I can however get around it by using "#{env.cwd}/.vagrant" instead of env.local_data_path but it feels a bit hackish.

Working workaround

def link_inventory_to_vagrant(folder)
    env = Vagrant::Environment.new(vagrantfile_name: '')
    static_inventory = File.join(env.cwd, folder)
    ansible_inventory = File.join(
        "#{env.cwd}/.vagrant",
        '/provisioners/ansible/inventory'
    )
    FileUtils.mkdir_p ansible_inventory
    Dir[File.join(static_inventory, '*')].each do |path|
        next if File.basename(path) == 'hosts' # skip linking hostnames
        FileUtils.ln_sf path, ansible_inventory
    end
end

link_inventory_to_vagrant 'provisioning/ansible/environments/dev'

[coreyoconnor]

The work around I used is:

~
ansible.raw_arguments = ["--inventory", YOUR_INVENTORY_PATH ]
~

[gildegoma]

Closing this issue as the workaround proposed by @coreyoconnor is a very easy way to go to satisfy this "rather unusual" case.