Molecule: Compatibility with Ansible devel (2.8) broken

[pilou-]

Issue Type

• Bug report

Since 7e92ff823e2bf97ee33c1b7ff87bb8acecb5ae77, molecule doesn't work with ansible devel version.

Error is:

        ==================================== ERRORS ====================================
        ____________________ ERROR collecting tests/test_default.py ____________________
        tests/test_default.py:6: in <module>
            os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
        /home/travis/build/ansible/molecule/.tox/ansibledevel-functional/lib/python3.7/site-packages/testinfra/utils/ansible_runner.py:153: in __init__
            self.cli.options = self.cli.base_parser(
        E   AttributeError: 'PlaybookCLI' object has no attribute 'base_parser'

Issue was reported by @maxamillion on #ansible-molecule channel:

@maxamillion | can anyone advise what I'm doing wrong here? https://paste.fedoraproject.org/paste/6Q4OJP4CQxfzGU0HzqTpog
@maxamillion | or what I might be missing                                                                               
@maxamillion | I *think* it's a bug in testinfra, but I'm not certain                                                   
       Pilou | maxamillion: it seems base_parser has been removed (7e92ff823e2bf97ee33c1b7ff87bb8acecb5ae77)            
      @bcoca | maxamillion: tanner was saying this last week, ansible changed the internal options parser               
      @bcoca | and molecule uses it directly, which now breaks with devel                                               
      @bcoca | you can still use with <=2.7                                                                             
@maxamillion | oh right, I vaguely remember that conversation                                                           

[singuliere]

I'll give this a try. Not sure to succeed but announcing anyways ;-)

[singuliere]

Reproduced locally (see the full output) with

TOXENV='ansibledevel-functional' tox -- -k 'test_command_init_role[docker]' test/functional/test_command.py

with the following diff on tox.ini (also proposed as an enhancement)

diff --git a/tox.ini b/tox.ini
index 8e0e3823..30932e68 100644
--- a/tox.ini
+++ b/tox.ini
@@ -30,7 +30,7 @@ extras =
     vagrant
 commands =
     unit: pytest test/unit/ --cov={toxinidir}/molecule/ --no-cov-on-fail {posargs}
-    functional: pytest test/functional/ {posargs}
+    functional: pytest {posargs:test/functional/}
     lint: flake8
     lint: yamllint -s test/ molecule/

[singuliere]

The corresponding testinfra issue is https://github.com/philpep/testinfra/issues/401

[decentral1se]

Thanks for investigating @singuliere. I've submitted https://github.com/ansible/molecule/pull/1733 to stop us getting failing builds on master during the cron runs that test against ansible-devel. Let's hope to see a fix in testinfra soon.

[singuliere]

@lwm this is indeed the wise thing to do :-) I proposed a pull request to testinfra but it is not trivial and I suspect it will be some time before this is fixed.

[webknjaz]

Basically testinfra uses private API which is not (and never was) guaranteed to be stable. The right way, as for now, is to call CLI. The long-term fix would be to use ansible-runner which (I heard so) is going to be ready at some point in future (aka probably soon).

[singuliere]

The right way, as for now, is to call CLI.

@webknjaz there is no way to implement the testinfra ansible backend with the ansible CLI. The only choice is to use the unstable API. Or to not provide support for ansible at all, which we would not want, right? :smiley:

[webknjaz]

Well, I just relayed what I heard from our internal slack. Didn't look into it personally.

[decentral1se]

I think a number of tools have struggled with this choice (API vs. CLI), including this one! See https://molecule.readthedocs.io/en/latest/faq.html#have-you-thought-about-using-ansible-s-python-api-instead-of-playbooks. I am watching ansible-runner closely (I even tried a little experiment myself and the API is coming along fine IMHO!

[decentral1se]

PR out for a fix in testinfra -> https://github.com/philpep/testinfra/pull/404

[decentral1se]

Further developments in https://github.com/philpep/testinfra/issues/401 :+1:

Luckily for us, it seems testinfra may be going in the direction of using ansible-runner!

[webknjaz]

Ref: https://github.com/ansible/molecule/pull/1983

[decentral1se]

OK people, we can move ahead with fixing this now. See https://github.com/philpep/testinfra/issues/401.

[decentral1se]

OK, we can close this off with https://github.com/ansible/molecule/pull/2034.

[decentral1se]

Ansible 2.8 dropped :scream: :scream: :scream:

Need to get the testinfra patch fixed and fix the CI to build 2.8 proper (without fails).

[hswong3i]

https://github.com/philpep/testinfra/issues/401#issuecomment-489434741

Hi, I just released 3.0.0 with the new ansible runner which should be compatible with ansible 2.8 please sse the changelog here: https://testinfra.readthedocs.io/en/latest/changelog.html

And testinfra is now 3.0.3:

• https://github.com/philpep/testinfra/releases

[decentral1se]

Reviews / comments / urgent demands welcome on https://github.com/ansible/molecule/pull/2034 !!!

[wildone]

I got around this by adding testinfra to travis.yml which installed testinfra 3.0+

• pip install testinfra molecule docker

[jonashackt]

Thanks @wildone - installing testinfra right before molecule (pip install testinfra molecule) provides a running workaround 👍 , until new Molecule version is available as pip package

[abeluck]

Using molecule's docker image here, and we need to add the following line to the dockerfile to make it work with ansible 2.8 and testinfra 3.x:

RUN pip install "ansible==2.8.0" "testinfra==3.0.4"

[decentral1se]

Thanks @abeluck, we have to make a new release soon ...

[lhoss]

Apart

pip install "ansible==2.8.0" "testinfra==3.0.4"

I also needed to bump cerberus (to v1.3.1) to avoid some schema errors.
Good that this bump was already done in master recently: https://github.com/ansible/molecule/pull/2103