Molecule: Specify Collection for Galaxy dependency

[artis3n]

Issue Type

• Feature request

Molecule and Ansible details

ansible --version && molecule --version

ansible 2.9.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/artis3n/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/artis3n/.local/share/virtualenvs/dev-setup-cQlga8yo/lib/python3.7/site-packages/ansible
  executable location = /home/artis3n/.local/share/virtualenvs/dev-setup-cQlga8yo/bin/ansible
  python version = 3.7.5 (default, Nov 24 2019, 17:46:35) [GCC 9.2.1 20191008]

molecule, version 2.22

Molecule installation method (one of):

• source
• [x] pip (pipenv)

Ansible installation method (one of):

• source
• [x] pip (pipenv)
• OS package

Desired Behavior

dependency:
  name: galaxy
  options:
    role-file: requirements.yml
    collection-file: requirements.yml

With a requirements.yml format following https://docs.ansible.com/ansible/devel/user_guide/collections_using.html#install-multiple-collections-with-a-requirements-file:

---

roles:
- src: gantsign.visual-studio-code
- src: artis3n.bitwarden_app

collections:
- name: artis3n.github

Molecule's galaxy dependency doesn't appear to support a collection-file similar to the role-file to install Galaxy Collections. I posed a question on Stack Overflow but based on the documentation I'm reasonably sure the functionality doesn't exist yet, so posting a feature request.

[greg-hellings]

It should be possible to just keep the same requirements.yml file, should it not? A call into the ansible-galaxy command twice, once with ansible-galaxy install -r <role-file> and a second time with ansible-galaxy collection install -r <role-file> should do the trick, right?

Per the Ansible documentation on the subject a plain array in the file will be construed as roles. I haven't tested this, but if it works as the docs claim then there's no need to introduce an additional file to Molecule. Just an additional call.

[artis3n]

Totally, I'd expect both roles and collections to be in the same requirements.yml file. Although as a user, I'd be confused why role-file is the parameter that handles collections. Not sure how that gets handled in a backwards-compatible way.

[greg-hellings]

@artis3n Well, if it /has/ to be changed, you could just change it to "requirements-file". Then also look for "role-file" and issue a deprecation warning if "role-file" is found. Use "requirements-file" if both are found and output a warning (or make it an error if they're both present).

But, in truth, there wouldn't be a NEED to change the name.

[artis3n]

Yeah there are absolutely a lot of technical ways to get this feature into molecule. In between requirements-file and collection-file I actually prefer collection-file because that opens up the possibility that someone _could_ use different requirements.yml. Not sure why they'd want that but molecule would get that extra functionality "for free." And it doesn't deprecate an existing option.

But I also agree with your first comment that that seems to duplicate the function of role-file, so yeah I do not know what would be best.

[greg-hellings]

Taking my hint from the Ansible docs -

If requirements.yml is just a list, e.g.

- foo.bar
- name: foo.baz
- ...

then ansible install -r <rolefile> will install all the roles listed in it. If it has keys in it, e.g.

roles:
  - foo.bar
  - name: foo.baz
collections:
  - derp.herp
  - ...

then ansible install -r <rolefile> will install only the values under the 'roles' key.

Likewise, ansible collection install -r <reqfile> will install only the values under the 'collections' key and ignore both the first format and anything under roles. So, we can leave it as roles-file and let Ansible do the heavy lifting for us, if we would like.

But now I come to a more difficult issue, as I'm looking through the Molecule code. The reason that 'role-file' is the preferred key is because the long name of the '-r' argument is '--role-file'. For the ansible-galaxy collection sub-command, the long form of '-r' is '--requirements-file'. Thus, to be consistent, we should be asking the user for requirements-file and let them set it to the same value if they want to, perhaps using YAML anchors.

I think, in my proposed PR, I will allow requirements-file and, if it is absent, fall back to using role-file.

[artis3n]

Given that I agree that requirements-file looks like the right thing

[greg-hellings]

As an update, I have code working in my local space for this. I'm working on updating unit and functional tests to accommodate the code changes I've made. Expect a PR within a day or two.

[greg-hellings]

PR #2609 is up. For me, at least, it supports what I need. It does not default to using the same file as the role-file argument. Prior to collections, a requirements.yml could just be a list of roles, e.g.:

- foo.bar  # This syntax will error on "ansible-galaxy collection install"
- foo.baz
- foo.yam

To install collections with ansible-galaxy you MUST indicate a collections key with appropriate names in a list. Optionally roles can be kept in the same file

collections:  # This is required for "ansible-galaxy collection install"
  - collection.foo
  - collection.bar
roles:  # This is optional and is understood by "ansible-galaxy install"
  - foo.bar
  - foo.baz

If I pointed collections at the same requirements.yml that is used for "ansible-galaxy install" calls currently, it would detect the file and could cause existing scenarios to break unnecessarily if they do not match the newer dictionary syntax.

So the PR looks, by default, for collections.yml and can be configured with the requirements-file key in your molecule.yml.

[greg-hellings]

Now that #2609 is merged, any chance we can get a release of molecule with this support?

[geerlingguy]

Bump / +1 here—I need to add a collection dependency to one of my molecule tests, and the docs currently have what's in #2609, so I thought I was going made having molecule keep breaking, until I found this issue and that PR, and found that Molecule's current release (3.0.3) doesn't have it...

Also, maybe this is a docs bug, but this code block https://github.com/ansible-community/molecule/blob/8205b8e338c779c1edd3e99e1823d7633178f447/molecule/dependency/ansible_galaxy/__init__.py#L36-L42 is not appearing in the generated docs at all on https://molecule.readthedocs.io/en/latest/configuration.html — I see the following:

[greg-hellings]

Hmmm, I wrote that portion and I'll admit to not being an rst expert. I spend most of my life writing in Markdown, so it's likely that I just completely botched the syntax for that.

Also, I could have sworn that 3.0.3 went out with #2609. At least I saw it mentioned in the release notes

[geerlingguy]

PR above fixes the docs formatting issue (and like you, I rarely touch rst ;).

[geerlingguy]

And you're correct, it looks like 3.0.3 has the fix; I was on 3.0.2 or some earlier 3.x release. I think this issue can be closed now.

[artis3n]

Agreed, thanks for getting this in @greg-hellings !