Awx: Cannot start Docker containers (Error starting project 404 Client Error: access denied for ansible/awx)

[eeepmb]

ISSUE TYPE

• Bug Report

SUMMARY

Running the AWX installer with the basic config ends in the error below. A few days ago everything was just working fine with the same setup.

Error starting project 404 Client Error: Not Found (\"b\"pull access denied for ansible/awx, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\"\")

ENVIRONMENT

• AWX version: 11.2.0
• AWX install method: docker on linux
• Ansible version: 2.9.9
• Operating System: 4.14.165-131.185.amzn2.x86_64 (Amazon Linux 2)
• Web Browser: none

STEPS TO REPRODUCE

Clone AWX repo and run AWX installer with basic configuration by sudo ansible-playbook -i inventory install.yml

EXPECTED RESULTS

AWX containers running.

ACTUAL RESULTS

Task Start the containers errored. Full traceback below.

ADDITIONAL INFORMATION

TASK [local_docker : Start the containers] ***************************
task path: /usr/share/data/awx/installer/roles/local_docker/tasks/compose.yml:45
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848 && echo ansible-tmp-1591268849.01-22646-108841225426848="` echo /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/docker/docker_compose.py
<localhost> PUT /root/.ansible/tmp/ansible-local-22090_ahKnV/tmpKjTH4y TO /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848/AnsiballZ_docker_compose.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848/ /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848/AnsiballZ_docker_compose.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/env python3 /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848/AnsiballZ_docker_compose.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1591268849.01-22646-108841225426848/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_docker_compose_payload_ymwlnr56/ansible_docker_compose_payload.zip/ansible/modules/cloud/docker/docker_compose.py", line 791, in cmd_up
  File "/usr/local/lib/python3.7/site-packages/compose/project.py", line 548, in up
    svc.ensure_image_exists(do_build=do_build, silent=silent, cli=cli)
  File "/usr/local/lib/python3.7/site-packages/compose/service.py", line 361, in ensure_image_exists
    self.pull(silent=silent)
  File "/usr/local/lib/python3.7/site-packages/compose/service.py", line 1250, in pull
    return progress_stream.get_digest_from_pull(event_stream)
  File "/usr/local/lib/python3.7/site-packages/compose/progress_stream.py", line 102, in get_digest_from_pull
    for event in events:
  File "/usr/local/lib/python3.7/site-packages/compose/service.py", line 1215, in _do_pull
    output = self.client.pull(repo, **pull_kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "errors": [], 
    "invocation": {
        "module_args": {
            "api_version": "auto", 
            "build": false, 
            "ca_cert": null, 
            "client_cert": null, 
            "client_key": null, 
            "debug": false, 
            "definition": null, 
            "dependencies": true, 
            "docker_host": "unix://var/run/docker.sock", 
            "files": null, 
            "hostname_check": false, 
            "nocache": false, 
            "project_name": null, 
            "project_src": "/root/.awx/awxcompose", 
            "pull": false, 
            "recreate": "smart", 
            "remove_images": null, 
            "remove_orphans": false, 
            "remove_volumes": false, 
            "restarted": true, 
            "scale": null, 
            "services": null, 
            "ssl_version": null, 
            "state": "present", 
            "stopped": false, 
            "timeout": 10, 
            "tls": false, 
            "tls_hostname": "localhost", 
            "validate_certs": false
        }
    }, 
    "module_stderr": "", 
    "module_stdout": "", 
    "msg": "Error starting project 404 Client Error: Not Found (\"b\"pull access denied for ansible/awx, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\"\")"
}

[Sheathan]

I'm running into the same issue since yesterday. Changes were pushed 23 hours ago, claiming to merge all containers into one AWX image, but now this docker image 404s.

[wsengkhyavong1]

I have exact same error message, just now.

[davidanet2002]

I am getting the same error on 3 different fresh installs: Centos 7 , Centos 8 , and ubuntu 20

[a-tyler]

Same here.
Arch Linux 5.6.15-arch1-1 #1 SMP PREEMPT Wed, 27 May 2020 23:42:26 +0000 x86_64 GNU/Linux

[wsengkhyavong1]

Any idea how to fix this error message?

[jkim-vcs]

Same since yesterday.

[shanemcd]

Looks like you are tracking the devel branch. Bold move! It's trying to pull an image that doesnt exist yet. This will be fixed in the next release (see first item).

Try to check out a specific tag or download a release from the GitHub Releases page.

[a-tyler]

I just ended up commenting out the dockerhub_base variable in awx/installer/inventory

[a-tyler]

@shanemcd
The devel branch is the default branch in github and following the documentation provided (https://github.com/ansible/awx/blob/devel/INSTALL.md#getting-started) fails.
Could you maybe suggest which tag is stable, change the documentation for the devel branch, or not set the devel branch as the default?

[shanemcd]

@a-tyler It appears maybe the documentation isnt clear enough:

We generally recommend that you view the releases page:

https://github.com/ansible/awx/releases

...and clone the latest stable release, e.g.,

git clone -b x.y.z https://github.com/ansible/awx.git

The command is doing -b x.y.z ... you should replace it with the latest release found at the link in the text above.

[a-tyler]

Thanks for the clarification. Don't know how I missed that.

[jkim-vcs]

I followed an instruction I found online and used the command below. Does this not get the latest stable version?

git clone --depth 50 https://github.com/ansible/awx.git

I see what's going on now. Thanks! I'm new to git/docker/awx as you can tell.

[davidanet2002]

so using the -b 11.2.0 did allow the containers to start atleast. finishing up.

[chendelin1982]

I'm running into the same issue since yesterday

[pravalli0630]

I'm running same issue since yesterday

[sivabgl]

change it in the inventory file

ansible/awx_tasksud

dockerhub_base=ansible

will work

[shanemcd]

You _can_ build from source, but just know that this is inherently risky. Bugs are being introduced and fixed between releases, and any given commit is a roll of the dice. I would caution you to do what's mentioned at the top of INSTALL.md, as mentioned here: https://github.com/ansible/awx/issues/7246#issuecomment-638959578

[eeepmb]

Thanks for the quick response and clarification!

Just one last question: Does this mean it was rather random luck that the installation worked the other day without specifing the branch/release?

[pravalli0630]

I tried with installing awx with specified version then i resolved my issue
.Thanks

On Mon, Jun 8, 2020, 3:16 PM eeepmb [e-mail] wrote:

Thanks for the quick response and clarification!

Just one last question: Does this mean it was rather random luck that the
installation worked the other day without specifing the branch/release?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ansible/awx/issues/7246#issuecomment-640496256, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AK5DCL4ZAIU64AUL6OTR6CLRVSXP3ANCNFSM4NSRRFGQ
.