Molecule: documentation: passing dynamic role variables to tests when runnning molecule verify

[decentral1se]

Following mixup on https://github.com/philpep/testinfra/issues/345.

Issue Type

• Documentation Request

Molecule and Ansible details

ansible 2.6.1
  config file = None
  configured module search path = ['/home/decentral1se/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/decentral1se/.local/share/virtualenvs/dye-imitation-UDPDxMAL/lib/python3.6/site-packages/ansible
  executable location = /home/decentral1se/.local/share/virtualenvs/dye-imitation-UDPDxMAL/bin/ansible
  python version = 3.6.5 (default, Jun 19 2018, 20:58:47) [GCC 6.3.0 20170516]

molecule, version 2.16.0

Molecule installation method (one of):

• pipenv

Ansible installation method (one of):

• pipenv

Issue

I know this could be batted away as something that is related to testinfra only, however, I am finding that I would like to ask here - would you accept a documentation PR that outlines a strategy for including role variables that you passed to your converge playbook, into your unit tests?

Here's what I am currently doing:

# molecule/default/playbook.yml
---
- name: Converge
  hosts: all
  vars_files:
    - testvars.yml
  roles:
    - role: my-role
      foobar: "{{ test_foobar }}"

Then in a molecule/default/testvars.yml, I have:

test_foobar: barfoo

Then, in my molecule/default/tests/conftest.py, I do the following:

import os

import pytest
from testinfra.utils.ansible_runner import AnsibleRunner

DEFAULT_HOST = 'all'

inventory = os.environ['MOLECULE_INVENTORY_FILE']
runner = AnsibleRunner(inventory)
runner.get_hosts(DEFAULT_HOST)


@pytest.fixture
def testvars(host):
    variables = runner.run(
        DEFAULT_HOST,
        'include_vars',
        'testvars.yml'
    )
    return variables['ansible_facts']

And in my tests, I can then:

def test_something(host, testvars):
    print(testvars['test_foobar'])

As far as I can see, this is the cleanest way (I've digged around in a lot of tickets) of matching up the dynamic variables that you pass to your role and your testinfra pytest tests.

If it isn't, please someone tell me :)

In any case, the main question remains - is there a place we can start to document this on the RTD setup?

[decentral1se]

Correct me if I am wrong, but it also appears that the best way to access default vars is:

@pytest.fixture
def defaultvars(host):
    variables = runner.run(
        DEFAULT_HOST,
        'include_vars',
        '../../defaults/main.yml'
    )

It would probably be good to write this down somewhere too.

[retr0h]

I'm not a huge fan of Molecule supporting this type of testing. Having a test which references a variable isn't really testing the variable. It makes the tests tightly coupled to the implementation, which may not catch expected outcomes.

You are free to do this if you choose, but Molecule doesn't want to accept a PR for this type of testing, b/c it's not really a paradigm we agree with.

Thank you for the suggestion @lwm

[decentral1se]

Having a test which references a variable isn't really testing the variable.

In my mind, it isn't about testing the variable but testing the role. And having the testinfra tests have a way of looking up the same inputs that the role accepted to verify things during the converge.

It makes the tests tightly coupled to the implementation, which may not catch expected outcomes.

If by the implementation, you mean the role? Or the molecule setup? Again, I'm just looking for a way to hand over the variables that go into my testing role to my testinfra tests.

it's not really a paradigm we agree with.

OK. However, the question remains, how do you pass variables to roles and look them up in the tests? If you are not doing that, what is your thinking behind your testing - you test around role variables?

Hope my use case is clear - I think a lot of people have it and it is confusing for people who want to do this kind of thing to come into the issues and search around. Also, there have been different techniques for doing this and some are old (useful for previous versions). An 'unsupported' FAQ could still do the job ...

[retr0h]

OK. However, the question remains, how do you pass variables to roles and look them up in the tests? If you are not doing that, what is your thinking behind your testing - you test around role variables?

That's a testinfra question. Not something I can really help with. We don't do it.

[RebelCodeBase]

I'm not a huge fan of Molecule supporting this type of testing. Having a test which references a variable isn't really testing the variable. It makes the tests tightly coupled to the implementation, which may not catch expected outcomes.

@retr0h, I can see your point when it comes to unit testing. And I agree that this is a testinfra issue. But even then it would be nice if jinja2 templates were resolved.

But when I use molecule and testinfra for integration or system testing then I'd like to have access to all role variables as well as those variables provided by molecule and testinfra. And I want my templates to be resolved.

I've created a pull request for testinfra and outlined a module I am currently working on:
https://github.com/philpep/testinfra/pull/462

[RebelCodeBase]

Here is a pytest plugin that exposes ansible variables and facts, gopass secrets and the ansible python api as pytest fixtures in testinfra tests for molecule: testaid