Molecule: Limit molecule tests to a specific docker image

[samrocketman]

Issue Type

• Feature request

Molecule and Ansible details

$ ansible --version
ansible 2.3.0.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

$ molecule --version
molecule, version 1.24.0

• Molecule installation method: virtualenv + pip
• Ansible installation method: virtualenv + pip

e.g.

virtualenv --python $(type -P python2.7) .venv
source .venv/bin/activate
pip install -U pip molecule ansible docker

Desired Behaviour

There's no way to execute molecule test with a specific docker image when multiple docker images are defined. Take for example the following YAML.

---
dependency:
  name: galaxy
driver:
  name: docker
docker:
  containers:
    - name: ubuntu-16.04
      image: ubuntu
      image_version: 16.04
      ansible_groups:
        - ubuntu
    - name: debian-jessie
      image: debian
      image_version: jessie
      ansible_groups:
        - debian
verifier:
  name: testinfra
molecule:
  ignore_paths:
    - .git
    - .venv

I would like to run my tests with, for example, only Debian Jessie. It would be nice to do something along the lines of:

molecule test --driver docker --container 'debian-jessie'

Actual Behaviour (Bug report only)

No combinations of options from molecule test --help give me the ability to limit which docker containers to launch.

$ molecule test --help
Usage: molecule test [OPTIONS]

  Runs a series of commands (defined in config) against instances for a full
  test/verify run.

Options:
  --driver TEXT                   Specificy a driver.
  --platform TEXT                 Specify a platform.
  --provider TEXT                 Specify a provider.
  --destroy [passing|always|never]
                                  Destroy behavior.
  --sudo / --no-sudo              Enable or disable running tests with sudo.
                                  Default is disabled.
  --help                          Show this message and exit.

[retr0h]

@samrocketman What would the use case for this be? I'm not so sure I see much value in adding another comand line option to test.

The developer workflow is generally to manually modify molecule.yml in the desired way while developing a role. Once the role is done with development, molecule.yml should represent what they wish to test.

So basically I recommend manually changing things during development phase. Otherwise, if you wish to change the container in CI I recommend this approach.

[samrocketman]

One use case is parallelizing Travis CI builds with matrix builds. (or any CI system)

Also, there doesn't have to be an additional option. I'm okay with repurposing an existing option like --platform which does nothing in the docker driver.

Or what about limiting via a shared driver property such as ansible_groups which isn't used as far as I know?

[retr0h]

The drivers unfortunately do not have consistency. Molecule v2 corrects this. If you wish to create a matrix for travis, I suggest using a combination of tox and ENV vars in molecule.yml. I'm not spending much effort adding new functionality to v1. V2 will be out soonish. However, the workaround I gave you my original response should get you going with tox.

[samrocketman]

Okay, thanks for letting me know. Another workaround is to symbolic link.

[retr0h]

Any reason you don't simply test all OSes together as a multi-node multi-distro test? What would the benefit of looping though each distro on it's own be?

Simply curious so I can better understand the community's use cases.

[samrocketman]

It's not looping through each. It runs them in parallel. So let's say you're testing 4 different OSes. You can launch each OS on a separate machine and test them all at once. The intent there is to speed up the build (link) by running independent parts in parallel on separate machines. Results are gathered when all four are done.

[mcg]

Found this issue while trying to do something similar. In my case I have a container config with some volume_mounts that do not work with my local macOS docker setup, but do with my Linux/Jenkins setup. I was searching for a way to selectively run one of these container configs that doesn't include or has an alternative volume_mounts locally. Then the other when running under CI.

[aabouzaid]

OK, I think anyone used Kitchen.ci would like to do the same style (which's actually reasonable).
So my workaround is for molecule v2 using converge with Ansible options to limit the execution to selected hosts.

in .travis.yml:

env:
  matrix:
    - INSTANCE: platform-instance01
    - INSTANCE: platform-instance02
script:
  - molecule converge -- --limit="localhost,${INSTANCE}"

And in you can edit the scenario converge_sequence as you like.
It's fine here with ansible-role-yourls, like this build.

However, I really would like to see --platform-name as part of molecule, it's really helpful :-)
Thanks.