molecule.yml environment variables

[mielas]

Issue Type

• Feature request

Molecule and Ansible details

ansible --version
ansible 2.4.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/gitlab-runner/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

molecule --version
molecule, version 2.2.1

• Molecule installation method: pip
• Ansible installation method: OS package

Problem

from http://molecule.readthedocs.io/en/latest/configuration.html:

Configuration options may contain environment variables. For example, suppose the shell contains VERIFIER_NAME=testinfra and the following molecule.yml is supplied.

tried to use some env variables e.g.:

---
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: centos-chrony-review-${foo}
    image: centos:latest
    command: /sbin/init
    privileged: yes

but it results with an error while doing the create playbook (it tries to create "centos-chrony-review-${foo}" container):

    TASK [Create molecule instance(s)] *********************************************
[...]
    <127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/gitlab-runner/.ansible/tmp/ansible-tmp-1507816543.65-231510959245136/docker_container.py; rm -rf "/home/gitlab-runner/.ansible/tmp/ansible-tmp-1507816543.65-231510959245136/" > /dev/null 2>&1 && sleep 0'
[...]
        "item": {
            "command": "/sbin/init", 
            "image": "centos:latest", 
            "name": "centos-chrony-review-${foo}", 
            "privileged": true
        }, 
        "msg": "Error creating container: 400 Client Error: Bad Request (\"{\"message\":\"Invalid container name (centos-chrony-review-${foo}test), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed\"}\")"
    }

    PLAY RECAP *********************************************************************
    localhost                  : ok=3    changed=2    unreachable=0    failed=1   

at the same time after setting static container name:

---
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: centos-chrony-review-test
    image: centos:latest
    command: /sbin/init
    privileged: yes

and running

molecule -s scenario converge

so molecule//.molecule directory is provisioned and changing molecule.yml again to:

platforms:
  - name: centos-chrony-review-${foo}

it interpretes env variable properly during the converge stage (it tries to use centos-chrony-review-test):

--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    fatal: [centos-chrony-review-test]: UNREACHABLE! => {"changed": false, "msg": "Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote temp path in ansible.cfg to a path rooted in \"/tmp\". Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp/ansible-tmp-1507817271.01-143009559961806 `\" && echo ansible-tmp-1507817271.01-143009559961806=\"` echo ~/.ansible/tmp/ansible-tmp-1507817271.01-143009559961806 `\" ), exited with result 1", "unreachable": true}

    PLAY RECAP *********************************************************************
    centos-chrony-review-test  : ok=0    changed=0    unreachable=1    failed=0

the question is: is there a way to use env variable in instance name? i've tried docker and openstack drivers with no luck

[retr0h]

This works for me. Do you have old create/destroy playbooks? Do the top of your create and destroy contain:

  vars:
    molecule_file: "{{ lookup('env', 'MOLECULE_FILE') }}"
    molecule_ephemeral_directory: "{{ lookup('env', 'MOLECULE_EPHEMERAL_DIRECTORY') }}"
    molecule_yml: "{{ lookup('file', molecule_file) | molecule_from_yaml }}"

The key part being molecule_from_yaml.

[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± git diff
diff --git a/test/scenarios/driver/docker/molecule/default/molecule.yml b/test/scenarios/driver/docker/molecule/default/molecule.yml
index 3788865..c4d33e3 100644
--- a/test/scenarios/driver/docker/molecule/default/molecule.yml
+++ b/test/scenarios/driver/docker/molecule/default/molecule.yml
@@ -8,7 +8,7 @@ lint:
   options:
     config-file: ../../../resources/.yamllint
 platforms:
-  - name: instance
+  - name: centos-chrony-review-${FOO}
     image: centos:latest
 provisioner:
   name: ansible

[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± export FOO=bar
[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± echo $FOO
bar
[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± molecule list
Instance Name             Driver Name    Provisioner Name    Scenario Name    Created    Converged
------------------------  -------------  ------------------  ---------------  ---------  -----------
centos-chrony-review-bar  Docker         Ansible             default          False      False
instance-1                Docker         Ansible             multi-node       False      False
instance-2                Docker         Ansible             multi-node       False      False

[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± molecule converge
--> Test matrix

└── default
    ├── create
    ├── prepare
    └── converge
--> Scenario: 'default'
--> Action: 'create'

    PLAY [Create] ******************************************************************

    TASK [Create Dockerfiles from image names] *************************************
    ok: [localhost] => (item=(censored due to no_log))

    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=(censored due to no_log))

    TASK [Build an Ansible compatible image] ***************************************
    changed: [localhost] => (item=(censored due to no_log))

    TASK [Create molecule instance(s)] *********************************************
    changed: [localhost] => (item=(censored due to no_log))

    PLAY RECAP *********************************************************************
    localhost                  : ok=4    changed=2    unreachable=0    failed=0

--> Scenario: 'default'
--> Action: 'prepare'

    PLAY [Prepare] *****************************************************************

    PLAY RECAP *********************************************************************

--> Scenario: 'default'
--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [centos-chrony-review-bar]

    TASK [molecule : Create /etc/molecule] *****************************************
    changed: [centos-chrony-review-bar]

    TASK [molecule : Create /etc/molecule/centos-chrony-review-bar] ****************
    changed: [centos-chrony-review-bar]

    PLAY RECAP *********************************************************************
    centos-chrony-review-bar   : ok=3    changed=2    unreachable=0    failed=0

[jodewey:~/git/molecule_2/test/scenarios/driver/docker] [venv] master(+1/-1)+ ± docker ps
CONTAINER ID        IMAGE                          COMMAND             CREATED             STATUS              PORTS               NAMES
5cccf31e7624        molecule_local/centos:latest   "sleep infinity"    43 seconds ago      Up 42 seconds                           centos-chrony-review-bar

[mielas]

that was it. at least create looks ok, didn't have time for more tests yet (destroy also needs the "patch")

but there is a question:

[gitlab-runner@centos molecule]$ molecule --version
molecule, version 2.2.1
[gitlab-runner@centos molecule]$ molecule init role --role-name role1
--> Initializing new role role1...
Initialized role in /home/gitlab-runner/molecule/role1 successfully.
[gitlab-runner@centos molecule]$ head -12 role1/molecule/default/create.yml 
---
- name: Create
  hosts: localhost
  connection: local
  gather_facts: False
  no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
  vars:
    molecule_file: "{{ lookup('env', 'MOLECULE_FILE') }}"
    molecule_ephemeral_directory: "{{ lookup('env', 'MOLECULE_EPHEMERAL_DIRECTORY') }}"
    molecule_scenario_directory: "{{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}"
    molecule_yml: "{{ lookup('file', molecule_file) | from_yaml }}"
  tasks:

why 2.2.1 is creating "from_yaml" and not "molecule_from_yaml" ? am i missing something in the docs?

[retr0h]

I don't see how that would be possible with 2.2.1. There isn't anything in the code which has from_yaml. The from_yaml changes were back from July in the Molecule 2 release candidates.
However, you're right.

[retr0h]

I don't see how that would be possible with 2.2.1.

Oh I see. I somehow missed updating the cookiecutter templates back in July and only updated the integration test plays. Thank you for catching that. Fix on the way.

[mielas]

great, keep up good work, tnx :)