Molecule: Silent failure: Boolean environment variable in `provisioner` causes `ansible-lint` to fail

[nkakouros]

Issue Type

• Bug report

Molecule and Ansible details

ansible 2.9.0.dev0
  config file = None
  configured module search path = ['/home/nikos/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/nikos/Projects/ansible/lib/ansible
  executable location = /home/nikos/Projects/ansible/bin/ansible
  python version = 3.7.3 (default, Mar 26 2019, 21:43:19) [GCC 8.2.1 20181127]
molecule, version 2.20.1

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• source

Detail any linters or test runners used:

dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: "instance-${MOLECULE_DISTRO:-ubuntu1604}"
    image: "geerlingguy/docker-${MOLECULE_DISTRO:-ubuntu1604}-ansible:latest"
    command: ${MOLECULE_INIT_CMD-""}
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro
    privileged: true
    pre_build_image: true
provisioner:
  name: ansible
  env:
    IN_MOLECULE: true
  lint:
    name: ansible-lint
verifier:
  name: testinfra
  enabled: false
  lint:
    name: flake8

Desired Behavior

I would expect the IN_MOLECULE environment variable to be accessible in the molecule playbook via a lookup.

Actual Behaviour

Molecule fails silently when trying to run ansible-lint with an exit code of 255:

[nikos@nikos-laptop nkakouros.elasticsearch]$ molecule lint
--> Validating schema /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix

└── default
    └── lint

--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/...
Lint completed successfully.
--> Executing Flake8 on files found in /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/playbook.yml...

output of running molecule --debug.

...
--> Executing Ansible Lint on /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/playbook.yml...
DEBUG: ANSIBLE ENVIRONMENT
---
ANSIBLE_CONFIG: /tmp/molecule/nkakouros.elasticsearch/default/ansible.cfg
ANSIBLE_FILTER_PLUGINS: /usr/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/nkakouros.elasticsearch/default/plugins/filters:/home/nikos/Projects/Ethical
  Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/plugins/filters
ANSIBLE_HOME: /home/nikos/Projects/ansible
ANSIBLE_LIBRARY: /usr/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/nkakouros.elasticsearch/default/library:/home/nikos/Projects/Ethical
  Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/library
ANSIBLE_ROLES_PATH: /tmp/molecule/nkakouros.elasticsearch/default/roles:/home/nikos/Projects/Ethical
  Hacking/EN2720/ansible/roles/services

DEBUG: MOLECULE ENVIRONMENT
---
MOLECULE_DEBUG: 'True'
MOLECULE_DEPENDENCY_NAME: galaxy
MOLECULE_DRIVER_NAME: docker
MOLECULE_ENV_FILE: /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/.env.yml
MOLECULE_EPHEMERAL_DIRECTORY: /tmp/molecule/nkakouros.elasticsearch/default
MOLECULE_FILE: /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/molecule.yml
MOLECULE_INSTANCE_CONFIG: /tmp/molecule/nkakouros.elasticsearch/default/instance_config.yml
MOLECULE_INVENTORY_FILE: /tmp/molecule/nkakouros.elasticsearch/default/inventory/ansible_inventory.yml
MOLECULE_LINT_NAME: yamllint
MOLECULE_PROJECT_DIRECTORY: /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch
MOLECULE_PROVISIONER_LINT_NAME: ansible-lint
MOLECULE_PROVISIONER_NAME: ansible
MOLECULE_SCENARIO_DIRECTORY: /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default
MOLECULE_SCENARIO_NAME: default
MOLECULE_VERIFIER_LINT_NAME: flake8
MOLECULE_VERIFIER_NAME: testinfra
MOLECULE_VERIFIER_TEST_DIRECTORY: /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/tests

DEBUG: SHELL REPLAY
ANSIBLE_CONFIG=/tmp/molecule/nkakouros.elasticsearch/default/ansible.cfg ANSIBLE_FILTER_PLUGINS=/usr/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/filters:/tmp/molecule/nkakouros.elasticsearch/default/plugins/filters:/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/plugins/filters ANSIBLE_HOME=/home/nikos/Projects/ansible ANSIBLE_LIBRARY=/usr/lib/python3.7/site-packages/molecule/provisioner/ansible/plugins/libraries:/tmp/molecule/nkakouros.elasticsearch/default/library:/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/library ANSIBLE_ROLES_PATH=/tmp/molecule/nkakouros.elasticsearch/default/roles:/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services MOLECULE_DEBUG=True MOLECULE_DEPENDENCY_NAME=galaxy MOLECULE_DRIVER_NAME=docker MOLECULE_ENV_FILE=/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/.env.yml MOLECULE_EPHEMERAL_DIRECTORY=/tmp/molecule/nkakouros.elasticsearch/default MOLECULE_FILE=/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/molecule.yml MOLECULE_INSTANCE_CONFIG=/tmp/molecule/nkakouros.elasticsearch/default/instance_config.yml MOLECULE_INVENTORY_FILE=/tmp/molecule/nkakouros.elasticsearch/default/inventory/ansible_inventory.yml MOLECULE_LINT_NAME=yamllint MOLECULE_PROJECT_DIRECTORY=/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch MOLECULE_PROVISIONER_LINT_NAME=ansible-lint MOLECULE_PROVISIONER_NAME=ansible MOLECULE_SCENARIO_DIRECTORY=/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default MOLECULE_SCENARIO_NAME=default MOLECULE_VERIFIER_LINT_NAME=flake8 MOLECULE_VERIFIER_NAME=testinfra MOLECULE_VERIFIER_TEST_DIRECTORY=/home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/tests

DEBUG: COMMAND
/usr/bin/ansible-lint -v --exclude=/tmp/molecule/nkakouros.elasticsearch/default /home/nikos/Projects/Ethical Hacking/EN2720/ansible/roles/services/nkakouros.elasticsearch/molecule/default/playbook.yml

If I replace IN_MOLECULE: true with IN_MOLECULE: 'true', it works.

[ssbarnea]

There is no such thing as a boolean environment variable, shell has only strings and that behavior is a feature, not a bug.

I remember this was added at some point in order to avoid cases where people did boolean comparison on environment variables.... which are likely to give unexpected results.

[nkakouros]

If it is a feature, could you at least inform the user that what they are doing is not going to work? A silent error is a scary and hard-to-deal-with error.

[ssbarnea]

I totally agreee, user should not need to call it with --debug to find out that. Maybe you should rephrase bug title/description to make it clear that the bug is about silent failure.

PRs to fix it is welcomed too.

[ssbarnea]

No longer valid in current versions.