Molecule: MacOS: molecule is not compatible with py37: weird error

[ssbarnea]

Issue Type

• Bug report

Molecule and Ansible details

Using molecule from master installed inside a tox virtualenv:

    TASK [Log into a Docker registry] **********************************************
    task path: /Users/ssbarnea/rdo/tripleo-ci/.tox/molecule/lib/python3.7/site-packages/molecule/provisioner/ansible/playbooks/docker/create.yml:8
    The full traceback is:
    Traceback (most recent call last):
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 96, in run
        items = self._get_loop_items()
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 215, in _get_loop_items
        convert_bare=False)
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/utils/listify.py", line 34, in listify_lookup_plugin_terms
        terms = templar.template(terms.strip(), convert_bare=convert_bare, fail_on_undefined=fail_on_undefined)
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/template/__init__.py", line 481, in template
        disable_lookups=disable_lookups,
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/template/__init__.py", line 686, in do_template
        myenv.filters.update(self._get_filters(myenv.filters))
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/template/__init__.py", line 302, in _get_filters
        plugins = [x for x in self._filter_loader.all()]
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/template/__init__.py", line 302, in <listcomp>
        plugins = [x for x in self._filter_loader.all()]
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/plugins/loader.py", line 435, in all
        module = self._load_module_source(name, path)
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ansible/plugins/loader.py", line 345, in _load_module_source
        module = imp.load_source(full_name, path, module_file)
      File "/Users/ssbarnea/.pyenv/versions/3.7.1/lib/python3.7/imp.py", line 171, in load_source
        module = _load(spec)
      File "<frozen importlib._bootstrap>", line 696, in _load
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 728, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/Users/ssbarnea/rdo/tripleo-ci/.tox/molecule/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/filters/molecule_core.py", line 23, in <module>
        from molecule import config
    ModuleNotFoundError: No module named 'molecule'

    fatal: [localhost]: FAILED! =>
      msg: Unexpected failure during module execution.
      stdout: ''

This is a very weird error because if I activate the same virtualenv and do do "import molecule" from python it works just fine. Still, when run like this it fails.

[decentral1se]

I run molecule under 3.7.2 and get no errors like this. I suspect it is something in your environment?

EDIT: Also, wouldn't the CI catch this?

[webknjaz]

@ssbarnea try finding out what env vars you have before running that. Do you have PYTHONPATH modified? etc.

[webknjaz]

And also you can try patching that /Users/ssbarnea/rdo/tripleo-ci/.tox/molecule/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/filters/molecule_core.py just to log env vars at that point as well. It should provide more clue on what's going on.

[webknjaz]

Oh, and I see you run this under macOS, is it reproducible under GNU/Linux as well?

[decentral1se]

It looks like this is a result of some weird interplay between Tox/Pyenv/Virtualenv and not Molecule.

Can we close this?

[ssbarnea]

Unless we get someone else on MacOS confirming that is working well with python 3.7, lets keep this open a litle bit more. I plan to investigate more but is not a no1 priority right now.

[jonbackhaus]

I’m having weird version incompatibilities when installing from pip on macOS. It’s downgrading a bunch of other packages I already have installed, including ansible-lint. Should I install via Homebrew? Or via pip from source?

[ssbarnea]

@jonbackhaus I totally agree with you on that aspect, this annoyed me too can caused problems: the pinning of dependency versions in molecule. Please create a new bug for this issue as is not specific to MacOS. I will try to see what we can do about it because pinning vs unpinning is a very controversial subject in general.

[decentral1se]

Wait a second, that shouldn't be happening with ansible-lint, we don't pin (note: we exclude lower version of 4.0.x due to a bug, IIRC) https://github.com/ansible/molecule/blob/master/requirements.txt#L2. Can you provide some logs @jonbackhaus? Thanks.

EDIT: Yes, probably a good idea for another ticket! :smiley_cat:

[jonbackhaus]

@decentral1se -- I figured it out: if you're trying to install from pip you're getting v2.19.0. Based on the commit history, that version has ansible-lint==3.4.23; see https://github.com/ansible/molecule/blob/0272bc0560df4f76e1c00617ca0e0dd8b5a46b58/requirements.txt#L1. Special thanks to wimglenn/johnnydep@5288894d27682adbc8c42d6f65c62067eaa36b34 for the awesome tool that helped me track down the issue!