Molecule: Validation for registry credentials (password) blocks passing actual credential value to underlying playbook

[watsonb]

Issue Type

• Bug report

Molecule and Ansible details

ansible --version
2.5.2

molecule --version
2.14.0

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Desired Behavior

The value of the environment variable for the Docker registry variable is passed to the docker create.yml playbook.

Actual Behaviour (Bug report only)

Assume I have exported some environment variables as follows:

export DOCKERUSER=foo
export DOCKERPASS=bar
echo $DOCKERUSER
># foo
echo $DOCKERPASS
># bar

Now, let us examine some platform iterations and their output. Note: to capture the output, I commented out the no_log attribute in the docker create.yml playbook.

Iteration 1:

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: $DOCKERUSER
        password: "{DOCKERPASS}"

Observation:
Molecule correctly parses the username attribute and passes the actual underlying value (foo) to the create.yml playbook. The password attribute validates, but Molecule passes the literal string "{DOCKERPASS}" to the create.yml playbook. This is the wrong password and the registry throws authentication error (HTTP 401 codes).
Output:

TASK [Log into a Docker registry] **********************************************
    failed: [localhost] (item={'image': u'chrismeyers-centos7', 'name': u'centos7', 'registry': {'url': u'my.docker.registry.io', 'credentials': {'username': u'foo', 'password': u'{DOCKERPASS}'}}}) => {"changed": false, "item": {"image": "chrismeyers-centos7", "name": "centos7", "registry": {"credentials": {"password": "{DOCKERPASS}", "username": "foo"}, "url": "my.docker.registry.io"}}, "msg": "Logging into my.docker.registry.io for user foo failed - 401 Client Error: Unauthorized (\"Get https://my.docker.registry.io/v2/: unauthorized: authentication required\")"}

Iteration 2:
Slight alteration of the username attribute to prove that Molecule can handle both the $VAR and ${VAR} style of environment variable references.

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: ${DOCKERUSER}
        password: "{DOCKERPASS}"

Observation:
Molecule correctly parses the username attribute and passes the actual underlying value (foo) to the create.yml playbook. The password attribute validates, but Molecule passes the literal string "{DOCKERPASS}" to the create.yml playbook. This is the wrong password and the registry throws authentication error (HTTP 401 codes).
Output:

TASK [Log into a Docker registry] **********************************************
    failed: [localhost] (item={'image': u'chrismeyers-centos7', 'name': u'centos7', 'registry': {'url': u'my.docker.registry.io', 'credentials': {'username': u'foo', 'password': u'{DOCKERPASS}'}}}) => {"changed": false, "item": {"image": "chrismeyers-centos7", "name": "centos7", "registry": {"credentials": {"password": "{DOCKERPASS}", "username": "foo"}, "url": "my.docker.registry.io"}}, "msg": "Logging into my.docker.registry.io for user foo failed - 401 Client Error: Unauthorized (\"Get https://my.docker.registry.io/v2/: unauthorized: authentication required\")"}

Iteration 3:
Adding a dollar sign ($) to the password attribute:

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: ${DOCKERUSER}
        password: "${DOCKERPASS}"

Observation:
Molecule correctly parses and passes username attribute but fails password validation.
Output:

ERROR: Failed to validate.

{'platforms': [{0: [{'registry': [{'credentials': [{'password': ["value does not match regex '^[{$]+[a-z0-9A-z]+[}]*$'"]}]}]}]}]}

Iteration 4:
Remove double quotes from password value

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: ${DOCKERUSER}
        password: ${DOCKERPASS}

Observation:
Molecule correctly parses and passes username attribute but fails password validation.
Output:

ERROR: Failed to validate.

{'platforms': [{0: [{'registry': [{'credentials': [{'password': ["value does not match regex '^[{$]+[a-z0-9A-z]+[}]*$'"]}]}]}]}]}

Iteration 5:
Remove double quotes AND curly braces from password value

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: ${DOCKERUSER}
        password: $DOCKERPASS

Observation:
Molecule correctly parses and passes username attribute but fails password validation.
Output:

ERROR: Failed to validate.

{'platforms': [{0: [{'registry': [{'credentials': [{'password': ["value does not match regex '^[{$]+[a-z0-9A-z]+[}]*$'"]}]}]}]}]}

In summary, to pass the regex validation, you have to wrap the value of the password attribute in double/single quotes. However, doing so passes the string literal of the password attribute to the underlying docker create.yml playbook, which is not the value of the referenced environment variable.

The hacky solution for me has been to delete the regex pattern matching lines (42 and 532)from molecule/model/schema_v2.py

Iteration 6:
Schema hack

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      url: "my.docker.registry.io"
      credentials:
        username: $DOCKERUSER
        password: $DOCKERPASS

Observation:
Molecule correctly parses the username and password attributes and passes the actual underlying values (foo and bar) to the create.yml playbook. I see the auths section of ~/.docker/config.json properly updated, and the image pulls down to my machine from my custom registry.
Output:

TASK [Log into a Docker registry] **********************************************
    changed: [localhost] => (item={'image': u'chrismeyers-centos7', 'name': u'centos7', 'registry': {'url': u'my.docker.registry.io', 'credentials': {'username': u'foo', 'password': u'bar'}}})

I truly appreciate the thoughtfulness behind forcing folks to store their sensitive credentials in a place that won't accidentally be committed to source control. However, I think the implementation needs a little more work. Kudos on the Molecule project thus far and keep up the good work.

[retr0h]

- name: centos7
    image: "chrismeyers-centos7"
    registry:
      credentials:
        password: "{DOCKERPASS}"

This syntax is wrong. Molecule is telling you not to use it.

The docs are telling you to use the following syntax:

  - name: instance
    hostname: instance
    image: image_name:tag
    registry:
      url: registry.example.com
      credentials:
        username: $USERNAME
        password: $PASSWORD
        email: [email protected]

Also, Molecule no longer requires you to pass username as an env var. Password is the only one which forces the env var.

[retr0h]

However, oddly I missed removing the regexp check from the normal validation. I'll fix that.

[retr0h]

This bug was reported and fixed in #1315. However, 2.15 is not released yet.

[watsonb]

Awesome, thanks!

[retr0h]

However, I did find an issue with #1315. I'll have 2.15 out later today.

[retr0h]

Fixed in #1332

[efpe]

Hi!
I didn't want to open a new issue but I have a problem when trying to use $CI_REGISTRY_PASSWORD for docker registry password in molecule.yml.

I think the validation still fails for passwords from the environment variable with underscores because the pre_validate_base_schema catches it.

platforms:
  - name: instance-1
    image: blabla
    registry:
      url: registry.example.com
      credentials:
        username: "${CI_REGISTRY_USER:-registry}" # This works as expected
        password: "$CI_REGISTRY_PASSWORD"
    privileged: True
    command: ""

Error message:

ERROR: Failed to validate.

{'platforms': [{0: [{'registry': [{'credentials': [{'password': ["value does not match regex '^[{$]+[a-z0-9A-Z]+[}]*$'"]}]}]}]}]}

Would you mind to take a look?

Thanks,
Viktor