Awx: requirements.yml is ignored on project sync

Created on 24 Dec 2020 · 15Comments · Source: ansible/awx

ISSUE TYPE

Bug Report

SUMMARY

ENVIRONMENT

AWX version: 16.0.0
- AWX install method: docker on linux
- Ansible version: 2.9
- Operating System: CentOS 8
- Web Browser: -

STEPS TO REPRODUCE

Create a repository with a roles/requirements.yml
Add the Repository as a Git Project on AWX
Sync the Project

EXPECTED RESULTS

AWX should sync the dependencies from the requirements.yml file

ACTUAL RESULTS

requirements.yml sync is skipped

TASK [detect roles/requirements.(yml/yaml)] ************************************
skipping: [localhost] => (item={'ext': '.yml'}) 
skipping: [localhost] => (item={'ext': '.yaml'})

ADDITIONAL INFORMATION

I tried to sync a repository with a requirements.yml and it is always skipped. Tried it with different AWX Versions (15.0.1, 16.0) and different git repos (Bitbucket, Gitlab, GitHub). Maybe the format of my requirements.yml is wrong, but doing a ansible-galaxy command on the awx server works just fine.

requirements.yml:

---
- src: git+https://github.com/al-lac/ansible-role-chrony.git
  version: master

or:

---
- src: git+https://oauth:<key>@gitlab.local/al-lac/ansible-role-chrony.git
  version: master

bug

Source

al-lac

All 15 comments

just my 5cents, but as far as i remember requirements are synced only on job execution. Sio you have to trigger a job from a template, and then awx will pull the requirements. Syncing project skips it.

supaflyster on 24 Dec 2020

I think it needs to be synced with the project, otherwise i get the following error when trying to run templates:

ERROR! the role 'ansible-role-chrony' was not found in /tmp/awx_2_wihiv2zf/project/roles:/tmp/awx_2_wihiv2zf/requirements_roles:/var/lib/awx/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/tmp/awx_2_wihiv2zf/project
The error appears to be in '/tmp/awx_2_wihiv2zf/project/chrony.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
  roles:
    - ansible-role-chrony
      ^ here

al-lac on 25 Dec 2020

You have in your role meta https://github.com/al-lac/ansible-role-chrony/blob/master/meta/main.yml
role_name: chrony
and you use it as
roles:
- ansible-role-chrony

So it can't obviously find it.
You either put ansible-role-chrony in meta or specify name: ansible-role-chrony in requirements.yml

supaflyster on 25 Dec 2020

Ah yeah i am not using that repository. I am using my gitlab server for that. I changed both repositories to use my gitlab server now, changed the name in meta and requirements.yml. But still get the same error.
Both repositories available here:
https://gitlab.iamroot.at/al-lac/ansible-role-chrony
https://gitlab.iamroot.at/root/ansible-base-roles

al-lac on 25 Dec 2020

Can confirm this behaviour.
Did a clean 16.0.0 install today and both requirements.yml in collections/ and roles/ getting skipped.

It gets skipped on project sync and when starting a job it also will fail with the same error message as shown above.
This leads to an unusable behaviour for us, as we can't even use community.general collection.

KornKalle on 28 Dec 2020

👍1

Hi @KornKalle, what is the last version it worked for you? I also tried it with version 15 and got the same behaviour.

al-lac on 29 Dec 2020

Last working Version for me was 11.1.0

KornKalle on 29 Dec 2020

We are using 15.0.1 (started with 1.0) and it was always working. I've just spun up clean 15.0.1 in vagrant, and added https://gitlab.iamroot.at/root/ansible-base-roles/ as a project, and run a template with ansible-role-chrony.yml playbook, targeting localhost and it downloaded chrony role and started executing it.
Screenshot 2020-12-29 at 11 02 58
Screenshot 2020-12-29 at 11 03 12

supaflyster on 29 Dec 2020

I just added the project and template exactly like you did on one of our 15.0.1 instances. I still get the error that the role was not found when starting the template. We used the docker compose install method for this server.

al-lac on 29 Dec 2020

Same still for me. It was still working pre-upgrade on 11.1.0, so I would rather like to get some information how we can reproduce this behavior.
I also used the docker compose install method on Debian Buster with docker 19.03.12

KornKalle on 30 Dec 2020

From 15.0 onwards, there has been a change made to how the sync works (instead of global to per-org), by default an org has no credentials hence it does not honor requirements.yml, you might want to try creating a dummy galaxy credential (it's required even if you don't use galaxy) and associating it to your org via the API as mentioned here https://github.com/ansible/awx/issues/8341

constreference on 4 Jan 2021

@constreference thanks a lot! That solved it for me.

I had to create a new (dummy) galaxy credential and associate it with my org via the api.

Get galaxy credential:

GET /api/v2/credentials/?credential_type__kind=galaxy
{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 11,
            "type": "credential",
            "url": "/api/v2/credentials/11/",
            "related": {
                "created_by": "/api/v2/users/8/",
                "modified_by": "/api/v2/users/8/",
                "organization": "/api/v2/organizations/2/",

Associate galaxy credential to org (returns 204 on success):

POST /api/v2/organizations/2/galaxy_credentials/
{
    "id": 11,
    "associate": true
}

This change should really be better documented somewhere. An option to do the association via the UI should be implemented soon as well.

al-lac on 4 Jan 2021

❤1

@ryanpetrello can you review the solution here and triage this issue? This could go in a couple directions ui_next/installer/etc.

AlanCoding on 4 Jan 2021

@AlanCoding yep, the need to explicitly associate a public Galaxy credential is a change we made in a recent version of awx - and @al-lac's comment is an example of how to do that correctly.