Molecule: Add `molecule init collection`

[decentral1se]

Issue Type

- Feature request

After #2342 we should move on to allowing ansible-galaxy to init collections for us. We have https://www.jeffgeerling.com/blog/2019/how-add-integration-tests-ansible-collection-molecule already. It's just a case of having ansible-galaxy do the work and then putting a molecule folder in.

Related to #2165.

[darmach]

I have some issues using molecule to test the module in a collection. Followed @geerlingguy article linked above, however molecule does not seem to pick up the module for me:

--> Action: 'converge'
--> Sanity checks: 'docker'
ERROR! couldn't resolve module/action 'my_test'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/Users/darmach/venvs/mytestrole2/mynamespace/mycollection/molecule/default/playbook.yml': line 16, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
  - name: Test the my_test module.
    ^ here

ERROR:
(mytestrole2 venv)[darmach@mbp] ~/venvs/mytestrole2/mynamespace/mycollection [4] $ tree
.
├── README.md
├── docs
├── galaxy.yml
├── molecule
│   └── default
│       ├── INSTALL.rst
│       ├── molecule.yml
│       ├── playbook.yml
│       └── tests
│           ├── __pycache__
│           │   ├── test_default.cpython-37-pytest-5.2.2.pyc
│           │   └── test_default.cpython-37.pyc
│           └── test_default.py
├── plugins
│   ├── README.md
│   └── modules
│       └── my_test.py
└── roles

8 directories, 10 files

Used following ansible and molecule versions:

pip install git+https://github.com/ansible/ansible.git@devel
pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple molecule==3.0a4.dev41

Are these correct? Am I doing something wrong?

[decentral1se]

@darmach please do not spam our issues (you posted the same on the other ticket). Use StackOverflow, the mailing list and IRC - all valid places to look for support. This is for tracking a feature for the project, not a support space. I'm marking your comment as off-topic.

[decentral1se]

We can move ahead this now that https://github.com/ansible/molecule/pull/2342 is in.

[tadeboro]

I am not sure if the molecule folder should live in the root of the Ansible collection. Upstream's recommendation is to place tests into the tests folder. More info can be found here (I know it is a bit sad that the only real testing-related documentation is session recording from AnsibleFest, but better this than nothing).

For example, in sensu.sensu_go collection, we placed molecule tests under tests/integration, because this is what molecule tests are at their core.

[decentral1se]

It's unclear where Ansible core are going and I'd rather not step on their toes. When there is a consensus for this tests folder convention from core then I would consider it.

[greg-hellings]

Personally I've taken to dropping the molecule tests inside of the individual roles that they are testing for. Is there a reason to not continue doing that?

[geerlingguy]

@greg-hellings - A number of collections don't have any roles... and some collections only have roles. Others are a mix of roles and modules (and someday possibly other things like playbooks too).

I find value in being able to use molecule with all three of these primary resources.

[ssbarnea]

Cleary this is something we do want to do. A PR is more than welcomed here. Ideally we should generate something that has a little bit of each mentioned parts, making easy for users to build on top of it, or to remove stuff they do not need.

If you have such an example repo, I can help converting it to a cookiecutter template.

[greg-hellings]

@geerlingguy I get that. I haven't yet had a need for a collection that's modules only but it's a very understandable workflow.

For all my roles I've taken to using tox inside of my collection to isolate testing. I even wrote a plugin for tox to help. It automatically generates a tox environment and sets an appropriate "molecule test -s foo" to run every scenario in its own configured environment. It also lets me filter and only run scenarios based on a single role (--ansible-role foo), a single scenario name (--ansible-scenario bar) or a single scenario driver (--ansible-driver openstack). This is currently at use within my two personal collections and our first collection coming out of RHQE's CI efforts.

I wonder if a similar type of thing, or even an enhancement to it, can help isolate molecule tests running elsewhere within a collection's space to test other components. Then a skeleton could be used to quickly configure appropriate tox options?

[Yajo]

IMHO molecule should support building and installing a local collection from a directory.

What I do right now is:

# molecule.yml
depencency:
  name: shell
  command: ansible-playbook molecule/default/dependency.yml
# ...


---
# dependency.yml
- name: install dependencies
  hosts: localhost
  tasks:
    - name: remove last build
      file:
        path:
          "{{ playbook_dir }}/collections/mynamespace-mycollection-{{ (lookup('file',
          '../../galaxy.yml')|from_yaml).version }}.tar.gz"
        state: absent

    - name: remove last collection installation
      file:
        path: "{{ playbook_dir }}/collections/ansible_collections/mynamespace/mycollection"
        state: absent

    - name: build tested collection
      command:
        argv:
          - ansible-galaxy
          - collection
          - build
          - --force
          - --output-path={{ playbook_dir }}/collections
          - "{{ playbook_dir }}/../.."

    - name: install tested collection
      command:
        argv:
          - ansible-galaxy
          - collection
          - install
          - --force
          - --collections-path={{ playbook_dir }}/collections
          - "{{ playbook_dir }}/collections/mynamespace-mycollection-{{ (lookup('file',
            '../../galaxy.yml')|from_yaml).version }}.tar.gz"

This way I instruct molecule to build and install the collection it is testing locally. However this feels kind of hacky; IMHO I should be able to just do this:

```yaml

molecule.yml

dependency:
name: galaxy

...

---

requirements.yaml

collections:

• name: mynamespace.mycollection

src: ../..

Although I gess this is more ansible galaxy's problem than molecule, right?

[ssbarnea]

I do feel your pain on this but there are some reasons which prevented us from doing it. If we install collections too easily during the testing we may pollute developer ansible installation with tested code.

One of the key features of molecule is that allows developer to test code in isolation, without risking to mess his system.

Basically we need to find a way to install these dependencies in a location that unique to current test.

For collections that are installed from outside we can expect them to be releases, so the danger is considerably less than on local code.

Yep, this is more of a galaxy issue than molecule, still I did not hear much from them. I am also aware of important changes that are present only on 2.10 branch, which is not released. My guess is that any improvement work will be delayed until 2.10 is released, and likely when it is we will support these feature only with 2.10.

It would be ideal to create galaxy tickets and link them to this one, so we can coordinate better.

[Yajo]

Makes sense.

One of the key features of molecule is that allows developer to test code in isolation, without risking to mess his system.

One of the things https://github.com/ansible-community/molecule/issues/2380#issuecomment-640176023 does is that collections will end up in ./molecule/$scenario/collections/ansible_collections. I did that for this same reason, just in case it helps...