Awx: Load Inventory from project source not possible if vault present

[computerlyrik]

ISSUE TYPE

• Bug Report

COMPONENT NAME

• API / UI

SUMMARY

Cannot read inventory file from project source if a password proteted vault is present

ENVIRONMENT

• AWX version: 1.0.7.2
• AWX install method: local docker
• Ansible version: 2.5.1
• Operating System: ubuntu 16.04
• Web Browser: Firefox

STEPS TO REPRODUCE

• create a ansible repository
• add something into inventory
• add a vault within group_vars/all
• Try to import inventory from project source

EXPECTED RESULTS

load inventory

ACTUAL RESULTS

1
    1.350 INFO     Updating inventory 2: webdev inventory
2
    1.360 INFO     Reading Ansible inventory source: /var/lib/awx/projects/_6-output_omitted/esansible_tower/hosts
3
Traceback (most recent call last):
4
  File "/usr/bin/awx-manage", line 9, in <module>
5
    load_entry_point('awx==1.0.7.2', 'console_scripts', 'awx-manage')()
6
  File "/usr/lib/python2.7/site-packages/awx/__init__.py", line 116, in manage
7
    execute_from_command_line(sys.argv)
8
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
9
    utility.execute()
10
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
11
    self.fetch_command(subcommand).run_from_argv(self.argv)
12
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
13
    self.execute(*args, **cmd_options)
14
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
15
    output = self.handle(*args, **options)
16
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 994, in handle
17
    self.is_custom)
18
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 245, in load_inventory_source
19
    is_custom=is_custom).load()
20
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 180, in load
21
    data = self.command_to_json(base_args + ['--list'])
22
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 163, in command_to_json
23
    self.method, proc.returncode, stdout, stderr))
24
RuntimeError: ansible-inventory failed (rc=4) with stdout:
26
stderr:
27
ERROR! Attempting to decrypt but no vault secrets found

ADDITIONAL INFORMATION

[nicolaibaralmueller]

Vault files are not supported in inventory. You need to encrypt strings and insert into variable content.
The template needs to have the vault password under credentials.

[computerlyrik]

@nicolaibaralmueller
I even do not want anything from the vault. It just fails if there is a vault present within the best-practice-directory-structure.

I also do not understand what you mean with "vault not supported IN inventory". We are using a inventory source from a simple stupid ini-like file.
Why does it not ignore the vault?

[computerlyrik]

@nicolaibaralmueller Please tell me how to use AWX if we have a fully running ansible repository according to best practices and just want to have our inventory within the repository as a file. So anybody can change the inventory by a simple commit.
And yes, the playbooks need secrets, so there is a vault which is totally irrellevant for the inventory.

Everything works great on commandline. Just AWX wants to "pre-read" the inventory and parse it to its database.

I just want to make the process ignore the vault because it fails and it is accordingn to you not supported.

[nicolaibaralmueller]

Ok, not completely sure what you mean without a sample of your inventory folder structure.

A file and a string can be encrypted. Only strings are supported by Tower.

Like this:

vault_variable: !vault |
$ANSIBLE_VAULT;1.1;AES256
1209371203710283710283120

We are not using ini files in our environment. Just yaml.

If you have any vaulted strings in your variables you need to add the credentials in the awx template under credentials.

[computerlyrik]

@nicolaibaralmueller I do not want to encrypt anything for the inventory. I just want to read a plain text inventory within an ansible project structure.
Please tell me how to do this with the Source-> Project source option within AWX if there is a vault (completely unused) present.

[computerlyrik]

If still not knowing where AWX fails, i may create a simple github repo with three files to demonstrate?

[nicolaibaralmueller]

@nicolaibaralmueller I do not want to encrypt anything for the inventory. I just want to read a plain text inventory within an ansible project structure.

Well it complain about a vault decrypt failure. Maybe run the job with verbosity debug .

[computerlyrik]

I want the the vault to be ignored, not used whatever. If AWX wants to parse i file it should do so. And not rely on a vault password

[nicolaibaralmueller]

I want the the vault to be ignored.

You can't ignore the vault. If one exists within the inventory, you need to provide a vault credentials for it.

[computerlyrik]

What do you mean with "vault within the inventory"???

[nicolaibaralmueller]

What do you mean with "vault within the inventory"???

You need to provide examples i order for anybody to help I think.

[computerlyrik]

Okay. I will show the setup on github-repo right now.

[computerlyrik]

@nicolaibaralmueller Here is the repository: https://github.com/computerlyrik/AWX-issue-2245

Just try to read the inventory from project source within AWX.

[computerlyrik]

How to reproduce on AWX:

• Create a project with this repository as SCM
• Create a inventory
• Add a inventory source from the above project and choose file inventory

Resulting in error:

1
    1.599 INFO     Updating inventory 4: ISSUE-Inventory
2
    1.623 INFO     Reading Ansible inventory source: /var/lib/awx/projects/_9__issue_2245/inventory
3
Traceback (most recent call last):
4
  File "/usr/bin/awx-manage", line 9, in <module>
5
    load_entry_point('awx==1.0.7.2', 'console_scripts', 'awx-manage')()
6
  File "/usr/lib/python2.7/site-packages/awx/__init__.py", line 116, in manage
7
    execute_from_command_line(sys.argv)
8
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
9
    utility.execute()
10
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
11
    self.fetch_command(subcommand).run_from_argv(self.argv)
12
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
13
    self.execute(*args, **cmd_options)
14
  File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
15
    output = self.handle(*args, **options)
16
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 994, in handle
17
    self.is_custom)
18
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 245, in load_inventory_source
19
    is_custom=is_custom).load()
20
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 180, in load
21
    data = self.command_to_json(base_args + ['--list'])
22
  File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 163, in command_to_json
23
    self.method, proc.returncode, stdout, stderr))
24
RuntimeError: ansible-inventory failed (rc=4) with stdout:
26
stderr:
27
ERROR! Attempting to decrypt but no vault secrets found

[computerlyrik]

on command line this will work with
ansible-playbook -i inventory playbook.yml

It is just the roundtrip AWX is going while parsing the inventory.

[nicolaibaralmueller]

@nicolaibaralmueller Here is the repository: https://github.com/computerlyrik/AWX-issue-2245

Just try to read the inventory from project source within AWX.

I would do the following:

Your inventory is not setup properly.
Inventory should be folder: /Inventory/somefoldername/hosts
Put server.example.com inside the hosts file.

rename group_vars/all to /Inventory/somefoldername/group_vars/all/vars.yml and insert the encrypted string inside vars.yml
As stated above you are using an encrypted file named all. This is not supported by awx tower. You can however use encrypted files from ansible. You need to use encrypted strings inside vars.yml

Example:

---

vault_test: vault_variable: !vault |
$ANSIBLE_VAULT;1.1;AES256
1209371203710283710283120

When running the playbook:
ansible-playbook -i /Inventory/somefolder/hosts playbook.yml --ask-vault-pass

[computerlyrik]

rename group_vars/all to /Inventory/somefoldername/group_vars/all/vars.yml and insert the encrypted string inside vars.yml
As stated above you are using an encrypted file named all. This is not supported by awx tower. You can however use encrypted files from ansible. You need to use encrypted strings inside vars.yml

Example:
--- vault_test: vault_variable: !vault | $ANSIBLE_VAULT;1.1;AES256 1209371203710283710283120

Then what is the alternative for ansible-vault edit group_vars/all to edit the vault secrets?
I do not want to mess up a working workflow right now. There are other guys who are managing the vault.

[nicolaibaralmueller]

rename group_vars/all to /Inventory/somefoldername/group_vars/all/vars.yml and insert the encrypted string inside vars.yml
As stated above you are using an encrypted file named all. This is not supported by awx tower. You can however use encrypted files from ansible. You need to use encrypted strings inside vars.yml
Example:
--- vault_test: vault_variable: !vault | $ANSIBLE_VAULT;1.1;AES256 1209371203710283710283120

Then what is the alternative for ansible-vault edit group_vars/all to edit the vault secrets?
I do not want to mess up a working workflow right now. There are other guys who are managing the vault.

I don't understand your question. Sorry. Try running the playbook manually with --ask-vault-pass

[computerlyrik]

As stated above you are using an encrypted file named all. This is not supported by awx tower. You can however use encrypted files from ansible. You need to use encrypted strings inside vars.yml

Sorry? It works perfectly if you run the playbook as AWX-Job with some other inventory and vault password set up....

[computerlyrik]

@nicolaibaralmueller Please answer me one question: How do i read the inventory file in AWX as "inventory from project source" from a repository like set up above?

[computerlyrik]

Inventory should be folder: /Inventory/somefoldername/hosts
Put server.example.com inside the hosts file.

This is exactly what made it work. Thank a lot.
https://github.com/computerlyrik/AWX-issue-2245/commit/d94ef652b3a27429417de6339ea834418f9111cc

[computerlyrik]

Probably references:
https://github.com/ansible/awx/issues/223
https://github.com/ansible/awx/issues/514
https://github.com/ansible/awx/issues/2110

[jomach]

So I'm experiencing the same problem on awx 2.8.2. On the hosts file I have no encrypted strings. I get.;

Parsed /var/lib/awx/projects/repository/infra/ansible/inventories/prod/hosts inventory source with ini plugin
ERROR! Attempting to decrypt but no vault secrets found

I think ansible should provide a env like ANSIBLE_VAULT_PASS instead of ANSIBLE_VAULT_FILE, this way we could create a custom Credential and pass it there.

[martinm82]

I am experiencing the same with problem. My ansible project structure looks like this:

playbooks/my-playbook.yml
roles/
inventory/
   common/
      group_vars/
          - all.yml
   teamA/
       group_vars/
          - all.yml
          - vault.yml <-- encrypted
      shared-files/
          - id_rsa.pub
          - id_rsa  <-- encrypted
   teamB/
      [...]

Locally I am running the playbook this way:

ansible-playbook -i inventory/common -i inventory/teamA playbooks/my-playbook.yml --ask-vault-pass

It works perfectly fine but when configuring this in AWX I get an error when the inventory source inventory/teamA is evaluated it throws the following error:

ERROR! Attempting to decrypt but no vault secrets found

Unfortunately I cannot define any vault credential for this particular inventory.

Any ideas how I should solve this problem in AWX? Is it actually a supported use case to use multiple inventories? In AWX I created an inventory and added two sources (inventory/common and inventory/teamA)

[nicolaibaralmueller]

@martinm82 AWX doesn't support vault files. You have to an encrypted string inside the all.yml inventory files.

[martinm82]

Thanks @nicolaibaralmueller for the quick reply. That is really unfortunate as that means I cannot use AWX at all.
I could potentially fix the problem with vault.yml but we as well encrypt private ssh keys that are used by Ansible during the ssh connection. Changing this would mean we need to store the content of those files in strings and unnecessary store it back to a file.

Are there no plans to support vault files on AWX side? Vault files are quite often used and very well supported by Ansible itself.

[nicolaibaralmueller]

@martinm82 No you encrypt a string with below command.

ansible-vault encrypt_string <secret> --ask-vault-pass

Use that output and insert as a variable inside your inventory file. Example:

vault_user_ssh_key: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          32386664633634343663643366633337383338333264646539343738313765393838373533346161
          6430633462356234633931666634643439613833386162300a656566343739653931656137666336
          34356166346563303830303565303437303930346264323766363561353466313365383861653734
          3837633063353733300a653066376437343631313966323633383533303333366434656238646238
          3130

You have to add a vault credential with the decrypt secret in AWX and add it to the template. You can then just make a reference to the variable vault_user_ssh_key where you need it.

[martinm82]

I understand it but that means I have to encrypt whole files as strings now. Would be great if AWX supports the same features as Ansible.

But will try to migrate all vault files to vault strings

[nicolaibaralmueller]

I understand it but that means I have to encrypt whole files as strings now. Would be great if AWX supports the same features as Ansible.

But will try to migrate all vault files to vault strings

@martinm82 We had to do the same here. On the positive side, it is actually more simple to maintain as you have an overview of encrypted vars in the inventory instead of one big encrypted file.

[wenottingham]

Are there no plans to support vault files on AWX side? Vault files are quite often used and very well supported by Ansible itself.

A whole vault file for an inventory would need to either be:
a) decrypted and stored as plaintext in the API/UI
b) decrypted, processed, and arbitrarily re-encrypted in the API/UI with .
c) stored as a blob and actually re-templated to the filesystem along with the inventory definition to hopefully be processed by ansible right at runtime

a) is very bad from a security perspective. b) is a level of complexity we would prefer to not implement. c) gets complex if an inventory is shared across multiple playbook runs/repos that are using different vaults and vault passwords at the playbook level.