Molecule: Molecule Docker provisioning leads to package conflicts in Fedora

[n-batalha]

Issue Type

• Bug report

Molecule and Ansible details

$ ansible --version && molecule --version

ansible 2.7.2
  config file = None
  configured module search path = ['/home/nelson/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/nelson/.local/share/virtualenvs/reproduce_molecule_issue-PL5vMwDu/lib/python3.7/site-packages/ansible
  executable location = /home/nelson/.local/share/virtualenvs/reproduce_molecule_issue-PL5vMwDu/bin/ansible
  python version = 3.7.1 (default, Nov  5 2018, 14:07:04) [GCC 8.2.1 20181011 (Red Hat 8.2.1-4)]
molecule, version 2.19.0

Molecule and Ansible installation method: pipenv

Desired Behavior

I expected git to install successfully.

Actual Behaviour

A conflict occurs due to unrelated packages.

    TASK [example_bug : install Git and related tools in RedHat] *******************
    fatal: [instance]: FAILED! => {"changed": false, "failures": [], "msg": "Unknown Error occured: coreutils-single conflicts with coreutils-8.30-6.fc29.x86_64", "rc": 1, "results": []}

debug.log

Reproducibility

Minimal molecule role reproducing the issue: https://github.com/n-batalha/molecule_error_example

It basically occurs on a clean fedora:29 docker image with:

- name: install Git and related tools in RedHat
  dnf:
    name: ['git']
    state: present
    update_cache: true
  become: true

But installing the same packages on a new Docker container (without using molecule) results in no errors. I suspect some other system packages are installed by molecule, resulting in this conflict.

[webknjaz]

Is this reproducible if you install ansible and molecule via pip or yum for example?

[webknjaz]

https://github.com/ansible/molecule/blob/080621b35476487b8e76b9ddb7da569ca2f5f74a/molecule/cookiecutter/scenario/driver/docker/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D/Dockerfile.j2#L11-L12

[n-batalha]

Thanks, yes on my tests I installed those packages manually too.

pipenv afaik is just a pip wrapper, but I reran with pip and here is how to reproduce it all:

---

Molecule (pip/PyPI)

# in the root of https://github.com/n-batalha/molecule_error_example
$ python3 -m venv --system-site-packages ~/tmp/venv-molecule-test
$ . ~/tmp/venv-molecule-test/bin/activate
$ pip install molecule ansible docker-py
$ molecule test

Fails [today] with:

TASK [example_bug : install Git and related tools in RedHat] *******************
fatal: [instance]: FAILED! => {"changed": false, "failures": [], "msg": "Unknown Error occured: libxcrypt-common <= 4.3.3-3 is obsoleted by libxcrypt-4.3.4-1.fc29.x86_64", "rc": 1, "results": []}

---

Docker

In a manually provisioned container this works.

$ docker run --entrypoint /bin/bash -i -t fedora:29
[docker]$ dnf --assumeyes install python sudo python-devel python2-dnf bash && dnf clean all
[docker]$ dnf --assumeyes install git

[webknjaz]

@n-batalha and what's in your molecule/ folder?

[webknjaz]

Oh.. I see. Let me try that myself.

[webknjaz]

I'm also getting fatal: [instance]: FAILED! => {"changed": false, "failures": [], "msg": "Unknown Error occured: libxcrypt-common <= 4.3.3-3 is obsoleted by libxcrypt-4.3.4-1.fc29.x86_64", "rc": 1, "results": []}

[n-batalha]

Small disclosure, one file I noticed I changed from the molecule init generated role when I was trying to fix errors myself was the boilerplate Dockerfile.j2.

I now placed it as the source one:

https://github.com/ansible/molecule/blob/080621b35476487b8e76b9ddb7da569ca2f5f74a/molecule/cookiecutter/scenario/driver/docker/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D/Dockerfile.j2#L11-L12

I just add sudo to the dnf installed packages, as without it gives me:

fatal: [instance]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: sudo: command not found\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 127}

Adding sudo takes us to the same error you reproduced.

[webknjaz]

So I've injected a "pause" task into your role and tried ad-hoc module runs against the container created. It looks like the problem is with update_cache: true:

$ ansible all -m dnf -a 'name=git state=present' --connection docker -i f259df0e0888,
f259df0e0888 | CHANGED => {
    "changed": true,
    "msg": "",
    "rc": 0,
    "results": [
        "Installed: git"
    ]
}

$ ansible all -m dnf -a 'name=git update_cache=yes state=present' --connection docker -i f259df0e0888,
f259df0e0888 | FAILED! => {
    "changed": false,
    "failures": [],
    "msg": "Unknown Error occured: libxcrypt-common <= 4.3.3-3 is obsoleted by libxcrypt-4.3.4-1.fc29.x86_64",
    "rc": 1,
    "results": []
}

[webknjaz]

Now, I've spawned the new container out of the image stored in my local registry, but w/o executing a Molecule process:

$ docker run --rm --name molecule_local-fedora -it molecule_local/fedora:29 bash
[root@171d26c4da97 /]#

And ran the module against that one:

$ ansible all -m dnf -a 'name=git update_cache=yes state=present' --connection docker -i 171d26c4da97,
171d26c4da97 | FAILED! => {
    "changed": false,
    "failures": [],
    "msg": "Unknown Error occured: libxcrypt-common <= 4.3.3-3 is obsoleted by libxcrypt-4.3.4-1.fc29.x86_64",
    "rc": 1,
    "results": []
}

$ ansible all -m dnf -a 'name=git state=present' --connection docker -i 171d26c4da97,
171d26c4da97 | CHANGED => {
    "changed": true,
    "msg": "",
    "rc": 0,
    "results": [
        "Installed: git"
    ]
}

This proves that during testing molecule doesn't break anything.

[n-batalha]

Great, thanks also for sharing how you debug! I'll use this to triage better or fix issues upstream in the future.

A user still needs to have roles to provision machines with update_cache=yes but closing as it's not a molecule issue.

edit: I was going to debug further and maybe reopen in ansible/ansible but I see now you can transfer issues

[webknjaz]

Let's try it against the upstream fedora:29 image from Docker Hub:

➜ docker run --rm --name fedora-29 -it fedora:29 bash         
[root@2aa2ed5f9756 /]# which platform-python
bash: which: command not found
[root@2aa2ed5f9756 /]# python2
bash: python2: command not found
[root@2aa2ed5f9756 /]# python3
Python 3.7.1 (default, Oct 23 2018, 18:19:07) 
[GCC 8.2.1 20180801 (Red Hat 8.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
[root@2aa2ed5f9756 /]# whereis python3
python3: /usr/bin/python3 /usr/bin/python3.7 /usr/bin/python3.7m /usr/lib/python3.7 /usr/lib64/python3.7 /usr/include/python3.7m

First attempt fails because we haven't set up the correct interpreter (Fedora 29 only has Python 3 by default):

$ ansible all -m dnf -a 'name=git update_cache=yes state=present' --connection docker -i fedora-29,
fedora-29 | FAILED! => {
    "changed": false,
    "module_stderr": "/bin/sh: /usr/bin/python: No such file or directory\n",
    "module_stdout": "",
    "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error",
    "rc": 127
}

But when we do, the issue is the same:

$ ansible all -m dnf -a 'name=git update_cache=yes state=present' -e ansible_python_interpreter=/usr/bin/python3 --connection docker -i fedora-29,
fedora-29 | FAILED! => {
    "changed": false,
    "failures": [],
    "msg": "Unknown Error occured: libxcrypt-common <= 4.3.3-3 is obsoleted by libxcrypt-4.3.4-1.fc29.x86_64",
    "rc": 1,
    "results": []
}

What if we don't update cache?

$ ansible all -m dnf -a 'name=git state=present' -e ansible_python_interpreter=/usr/bin/python3 --connection docker -i fedora-29,       
fedora-29 | CHANGED => {
    "changed": true,
    "msg": "",
    "rc": 0,
    "results": [
        "Installed: git"
    ]
}

[webknjaz]

@n-batalha I've reopened it because it looks like it might be an Ansible Engine's dnf module issue. Maybe it's an issue with the dnf python API.

Anyway, I'm moving this under ansible/ansible, because it belongs there and GitHub now allows to move issues between repos (https://help.github.com/articles/transferring-an-issue-to-another-repository/). We'll also need to modify the original issue description though (to make @ansibot happy).

[webknjaz]

Hm.. It looks like GH doesn't let me transfer issues...

cc @gundalow can you do this?

[webknjaz]

I've checked down to fedora 24:

• 24-25 failing because they don't have update_cache method at all
• 26-27 work well
• 28+ fail with the originally reported error

[webknjaz]

@n-batalha since we cannot move this issue would you please fill in a new one under ansible/ansible referring this info?

[webknjaz]

I've done this myself. Closing this issue therefore. @n-batalha please communicate via ansible/ansible#49060 wrt further investigation.