Molecule: [FeatureRequest] Allow --limit with all molecule actions

[rockandska]

Issue Type

• Feature request

Molecule and Ansible details

ansible 2.5.5
molecule, version 2.19.0

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Desired Behavior

As asked by other people in #1471 , #1122, #827 , I think that molecule need to be able to filter the platforms on which the the tests are launched, based on what is provided into platforms key inside molecule.yml.

We indeed need to test multi-platform, but sometimes, when you know that the problem is only on one platform, need to only test this one before re-run the tests on all platforms.

As state in a comment by @retr0h , converge can already have this kind of comportment by specify molecule converge -- --limit instance-1 but it seems it not works with other options as :

• check
• create
• destroy
• idempotence
• list
• login
• prepare
• side-effect
• test
• verify

In general, having the possibility to specify provisioner options on the command line regardless the action will be the best.

This kind of behaviour was asked multiple times and will be really handy for people who works with molecule and don't get why it was closed every time it was asked.

Could you please reconsidering the addition of this feature ?

Thanks,
Best regards,

[ssbarnea]

I was expecting to have a parameter like -p|--platform that would allow to limit on which platforms to run the current scenario. Still, i was not able to spot this yet.

While (re)implementing this lost feature consider it in a way that could different provisioners to use it. I can fully understand that intitially it will be implemented only by some (docker?) and that is ok.

[rockandska]

Glad to hear it :smile: , but it will limit the options to pass to the provisioner and the users will not have access to other options as --tags etc.. and in case molecule add a new provisioner in the future, it will require to have many options, and some not available to all provisioners.

But having provisioner options pass to all command through 'molecule [command] -- [provisioner-options] ...' could be maybe simpler to implement and will allow to not be stick to actual / future options of the provider ?

[decentral1se]

This kind of behaviour was asked multiple times and will be really handy for people who works with molecule and don't get why it was closed every time it was asked.

I believe it was a question of resources and feature scope for previous maintainers. Now that this project is under Ansible proper, we should have more of both :) Thanks for raising this issue, looks useful to me.

[ssbarnea]

Here is another use case that I discovered which start to look more like a deal braker: assuming that your projects supports 2 platforms aaa and bbb and you want to add support for ccc, this being a big efforth.

The only workable way to do it is to add a non-voting job that tests the new platform in parallel with the old ones, allowing people to gradually implement new changes without affecting the rest.

From CI perspective this means that 3 different commands to be run, one for each platform and executed by different build workers: likely they would look like molecule test --all ....but how to limit the platforms for each of them? Running all is useless for this case as we alread know that one of them will fail for a long time.

[rgarrigue]

For consistency's sake, I'll add there's this --host in login command, for example molecule login --host molecule-common-debian9-systemd that should be merged

As ssbarnea said, I'ld expect a -p|--platform (or --limit) on every command allowing to further refine the already existing --scenario.

[TomaszKlosinski]

Is it possible to limit just verify (testinfra)? I'm testing an haproxy role with some dummy prometheus and jenkins containers (to test the load balancing). Now, I'd like to run my testinfra verification script against haproxy containers but not the dummy prometheus and jenkins ones. How can I do that?

[beenje]

@TomaszKlosinski, with testinfra it's easy to limit the tests to a specific group or node.
The get_hosts method takes a group name or hostname in parameter:

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('<host or group>')

[TomaszKlosinski]

Thanks for the help.

[ssbarnea]

If someone takes time to create a PR on this, I will be the first to review and help merging it. I was planning to add it myself but it seems that I do have a long queue of things that are more pressing than this.

[danielhoherd]

I'm happy to see this discussion continuing from past closed tickets. My use case for having a --platform flag is that I want to be able to run the exact same tests (ie: the same scenario) on many platforms, but I want the ability to spawn parallel CI jobs for this. Without the ability to target a single platform found in molecule.yml, I have to resort to creating new scenarios, which causes duplicate code and violates DRY.

[ssbarnea]

Did anyone made any efforths towards addressing this feature request? I am asking the pressure to implement it becomes bigger and bigger. As stated before the creating multiple scenarios does not scale. Also using a single overridable platform cripples the UX by preventing the users from running all plaforms.

[rshad]

Hi all!

This is also related to #1471 , #1122, #827

We got some issues when trying to design our tests to provide the possibility to select a determined platform to run a test in. Molecule 2.x provides nothing about a similar feature.

In order to pass this obstacle, I designed a temporary solution until Molecule gets this feature added.

My idea is based on modifying molecule.yml file each time we want to run molecule test with a specified image or platform. In my case, I have a similar configuration for each platform but the name and the image values differ from one to another.

For example, imagine I have the following configuration:

platforms:
  - name: platform_name
    image: image_name
    command: /sbin/init
    ulimits:
      - nofile:262144:262144
    privileged: true
    memory_reservation: 2048m

My script first print a list of available images, and request from the user to select an image

images=( "solita/ubuntu-systemd:bionic" "solita/ubuntu-systemd:xenial" "milcom/centos7-systemd" "ubuntu:trusty" "centos:6" )

echo "Please select an image. "

select IMAGE in "${images[@]}";
do
     echo "You picked $IMAGE ($REPLY)"
     break
done

Terminal Output

vagrant@ansible:~/wazuh-ansible$ ./run_cluster_mode.sh
Please select an image.
1) solita/ubuntu-systemd:bionic  4) ubuntu:trusty
2) solita/ubuntu-systemd:xenial  5) centos:6
3) milcom/centos7-systemd

I also have an array with each platform name I want to associate to each image

platform=( "bionic" "xenial" "centos7" "trusty" "centos6" )

Once the image is selected then using sed I replace both variables name and image in molecule.yml, taking into account that we should have a backup of molecule.yml, in my case it's called molecule.yml.template

index=$(($REPLY - 1))

cp "molecule.yml.template" "molecule.yml"

sed -i "s|image_name|${images[$index]}|g" "$i/molecule.yml"
sed -i "s/platform_name/${platform[$index]}/g" "$i/molecule.yml" 

Once my moldecule.yml is prepared, then I run my Molecule test:

molecule test --destroy=never

Or if you're working with Pipefile, then

pipenv run test

Where my Pipfile looks like

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
docker-py = "*"
ansible = "==2.7.13"
molecule = "==2.20.2"

[dev-packages]

[requires]
python_version = "2.7"

[scripts]
test ="molecule test --destroy=never"
worker ="molecule test -s worker --destroy=never"
agent ="molecule test -s wazuh-agent --destroy=never"
elasticsearch ="molecule test -s elasticsearch  --destroy=never"
kibana ="molecule test -s kibana --destroy=never"

# Verify ..
verify ="molecule verify"
verify_worker ="molecule verify -s worker"
verify_agent ="molecule verify -s agent"
verify_elasticsearch ="molecule verify -s elasticsearch"
verify_kibana ="molecule verify -s kibana"

# Destroy ..
destroy ="molecule destroy"
destroy_worker ="molecule destroy -s worker"
destroy_agent ="molecule destroy -s agent"
destroy_elasticsearch ="molecule destroy -s elasticsearch"
destroy_kibana ="molecule destroy -s kibana

Please find the complete script Here! .

Kind regards,

Rshad

[rshad]

@ssbarnea I'll try to dedicate a time to get this feature added in Molecule.