Awx: Fails to detect roles/requirements.yaml

[Aethylred]

ISSUE TYPE

• Bug Report

SUMMARY

When a project includes a roles/requirements.yaml file it is not detected or processed:

TASK [detect requirements.yml] *************************************************
ok: [localhost, -> localhost] => {"changed": false, "stat": {"exists": false}}

It looks like the task to detect the file looks specifically for requirements.yml and not also requirements.yaml.

This is contrary to the YAML offical extension: https://yaml.org/faq.html

(Generally because both .yaml and .yml are in common usage, they should be considered interchangeable as different people like different things for their own reasons. I'd expect AWX/Tower to check and use a file with either extension, and throwing an error if it finds both)

ENVIRONMENT

• AWX version: 3.5.2
• AWX install method: Tower
• Ansible version: 2.8
• Operating System: CentOS 7
• Web Browser: Chrome

STEPS TO REPRODUCE

Create an AWX/Tower role and specify requirements in roles/requirements.yaml and clone into a project

rename to requirements.yml and it works.

EXPECTED RESULTS

I expected that the roles specified in requirements.yaml would be installed

ACTUAL RESULTS

The file requirements.yaml was ignored and roles were not installed.

ADDITIONAL INFORMATION

We use Ansible Tower, but I think AWX has this issue too.

[ryanpetrello]

Agreed, it sounds like Ansible itself supports both:

https://docs.ansible.com/ansible/latest/reference_appendices/galaxy.html

Beginning with Ansible 1.8 it is possible to install multiple roles by including the roles in a requirements.yml file. The format of the file is YAML, and the file extension must be either .yml or .yaml.

[ryanpetrello]

@AlanCoding @chrismeyersfsu we should probably verify that this also works properly for new collections support in https://github.com/ansible/awx/pull/4265

https://github.com/ansible/awx/pull/4265/files#diff-f069d8ac794ad13606442d2194490282R141

[AlanCoding]

Yes, it needs to be addressed in a number of places.

(awx) bash-4.2$ grep -r "requirements\.yml" awx/main/
awx/main/tasks.py:            galaxy_req_path = os.path.join(project_path, 'roles', 'requirements.yml')
awx/main/tasks.py:            galaxy_collections_req_path = os.path.join(project_path, 'collections', 'requirements.yml')
awx/main/conf.py:    help_text=_('Allows roles to be dynamically downloaded from a requirements.yml file for SCM projects.'),
awx/main/conf.py:    help_text=_('Allows collections to be dynamically downloaded from a requirements.yml file for SCM projects.'),

(awx) bash-4.2$ grep -r "requirements\.yml" awx/playbooks/
awx/playbooks/project_update.yml:# roles_enabled: Allow us to pull roles from a requirements.yml file
awx/playbooks/project_update.yml:      - name: detect requirements.yml
awx/playbooks/project_update.yml:        stat: path={{project_path|quote}}/roles/requirements.yml
awx/playbooks/project_update.yml:      - name: fetch galaxy roles from requirements.yml
awx/playbooks/project_update.yml:        command: ansible-galaxy install -r requirements.yml -p {{roles_destination|quote}}
awx/playbooks/project_update.yml:      - name: detect collections/requirements.yml
awx/playbooks/project_update.yml:        stat: path={{project_path|quote}}/collections/requirements.yml
awx/playbooks/project_update.yml:      - name: fetch galaxy collections from collections/requirements.yml
awx/playbooks/project_update.yml:        command: ansible-galaxy collection install -r requirements.yml -p {{collections_destination|quote}}

This is a perfectly valid request, but I do want to mention that it has always been this way.

[AlanCoding]

If this merges it will need docs changes

[Elyytscha]

we have here 55 repos with roles/requierements.yaml

if its named like this awx does not detect it

if its named roles/requierements.yml it gets detected and used

what will it be in new versions: .yaml or .yml or both?

when will be the next awx release which is able to detect roles/requierements.yaml?

best regards

[rebeccahhh]

@Elyytscha we are planning to merge a PR (#5439 ) in the coming days that will allow for both .yaml and .yml extensions to be detected. I expect it in the next minor release or the one after it, but you can track our progress at the PR above. Currently I am running the solution through testing to make sure it will function correctly. Hope that helps!

[kdelee]

Having trouble verifying

I added a roles/requirements.yaml file here https://github.com/ansible/test-playbooks/pull/163

Made a project pointed to that branch of a fresh build of devel, but although looks like it is getting the right revision, not seeing the role download:

TASK [Repository Version] ******************************************************
ok: [localhost] => {
    "msg": "Repository Version a417966b15d04697498f13638f2c869b7a576478"
}
PLAY [Install content with ansible-galaxy command if necessary] ****************
TASK [detect roles/requirements.(yml/yaml)] ************************************
skipping: [localhost] => (item={'ext': '.yml'}) 
skipping: [localhost] => (item={'ext': '.yaml'}) 

[kdelee]

Ok I figured out what I was doing wrong, I needed to add a galaxy credential to the organization. Now it is working. Closing this as verified. Filed https://github.com/ansible/tower/issues/4775 for docs updates and possibly some debug output or UI tool tip updates to help users find out about this behavior change.

[killua-eu]

I'm hitting the same issue with awx 16.0.0 .. I have galaxy credentials set up, but roles/requirements(yml/yaml) gets skipped regardless. Any tips or way to troubleshoot this?

[rebeccahhh]

hi @killua-eu are you getting skip messages for both .yml and .yaml extensions? It would look like the same output from this earlier comment.