Molecule: Feature idea: Add a new molecule command to perform check (dry run)

[conorsch]

Supporting --check mode for Ansible roles isn't trivial; you have to make sure any tasks that register variables are read-only and marked with always_run: true. Personally I'm a big fan of leveraging check mode, but I think the difficulty of testing it makes many folks writing Ansible roles leave it by the wayside. Molecule can help with that.

As low-hanging fruit during edits for #110, adding an additional (optional) stage for the test functionality would be great:

  test:
    # sequence of commands to run when performing `molecule test`
    sequence:
      - destroy
      - syntax-check
      - create
      - converge
      - idempotence
      - check
      - verify
      - destroy

Comments welcome.

[retr0h]

Implemented in #201.

[conorsch]

@retr0h The --check and --syntax-check arguments to ansible-playbook are distinct—looks like #201 adds the --syntax-check functionality as molecule check. That's fine, but it's not the same thing as --check, which performs a dry-run of the playbook.

[retr0h]

Opening

[retr0h]

@conorsch I think I'm going to need more details on this. I'm not entirely sure how it would be helpful.

If we are to add check into the test sequence. Where would it go? It doesn't make sense to run it after converge, since nothing would then change.

Since change always returns success, I don't see how it would be useful in the test sequence. I can implement the changed subcommand, but not add it to the test sequence. People can then do what they wish with it?

[conorsch]

If we are to add check into the test sequence. Where would it go? It doesn't make sense to run it after converge, since nothing would then change.

Adding _after_ converge actually does make sense, although it's not intuitive. Many roles use a command/register pragma to store a dynamic variable, then inspect that var in a subsequent task as part of conditional logic. By default, Ansible skips command tasks, because it can't know the severity of side-effects, so the subsequent tasks inspecting the var will fail in check mode. Having Molecule run a check mode after converge would be very useful to catch errors like this.

It's technically possible to support dry-runs of a first-time role run with Ansible, but I don't see many role authors striving for it. For example, get_url tasks will be skipped by default, and then any tasks referencing a file downloaded by get_url will fail, since the file doesn't exist. A reasonable order for running --check mode seems to be:

      - destroy
      - syntax
      - create
      - converge
      - idempotence
      - check
      - verify

If you run check before converge, I'd wager that most Ansible roles would fail, since folks simply don't plan for that use case.

[geerlingguy]

Thanks for this, and to anyone finding this in the future, you can add it to your sequence in molecule.yml with:

scenario:
  name: default
  test_sequence:
    - lint
    - destroy
    - dependency
    - syntax
    - create
    - prepare
    - converge
    - idempotence
    # Added --check run test.
    - check
    - side_effect
    - verify
    - destroy