Galaxy: Lots of broken dependencies due to `role_name` changing hyphens to underscores

[brtknr]

Bug Report

SUMMARY

The new naming convention has broken a lot of linked repos which need to be reverted or we would like recommendation for a future proof naming convention.

STEPS TO REPRODUCE

Yesterday, we noticed with one of our recent roles stackhpc.beegfs that the ansible-role- part was not being omitted as it previously did each time Travis CI build job triggered a notification API. Then, we noticed that adding role_name: beegfs to meta/main.yml ensured that the ansible-role- part was omitting. We made the fatal assumption that this change affected all our other repos so as a result, charged ahead with adding this line to about 30+ roles currently available on our Ansible Galaxy repository. We came under the impression that this was a way to guarantee that this would secure their name. Then we noticed that the repositories with hyphens in have now converted to underscores.

The situation is that now we have lots of broken dependencies that affect almost all our other repositories. Is there a way to revert this change? At the moment, we have in total 26 repositories that previously had hyphens in that need to go back to their original state. Despited removing the role_name from meta/main.yml file, it appears that this does not restore the previous state.

EXPECTED RESULTS

Either: Restore former hyphenated names to 26+ repos under stackhpc organisation.
Or: Recommend us a future proof naming scheme that is unlikely to change again in the near future as we observed in one of the issues that even the role_name variable is under discussions to be deprecated (https://github.com/ansible/galaxy/issues/1042, https://github.com/ansible/mazer/issues/62).

ACTUAL RESULTS

At the moment, there does not appear to be any way of going back as a lay user.

Sample import log for one of our roles showing the rename taking place irreversiable.

INFO Starting import: task_id=281479, repository=stackhpc/ansible-role-dell-powerconnect-switch
INFO Content search - Looking for file "apb.yml"
INFO Content search - Looking for top level role metadata file
WARNING [role] Missing 'dependencies' field in metadata.
INFO [role] Linting...
INFO [role] Linting OK.
INFO Updating repository name "dell-powerconnect-switch" -> "dell_powerconnect_switch"
INFO [ROLE: dell_powerconnect_switch] Created new Content instance: id=29348, content_type="role", name="dell_powerconnect_switch"
INFO [ROLE: dell_powerconnect_switch] Adding role metadata tags
INFO [ROLE: dell_powerconnect_switch] Adding role platforms
INFO Deleting Content instance: content_type=role, namespace=stackhpc-3269, name=dell-powerconnect-switch
INFO Updating repository versions...
Import completed

Many thanks.

[markgoddard]

There are a few key issues here.

As a consumer of Galaxy roles, there is no smooth transition from the old names to the new names. All existing playbooks and Galaxy requirements.yml files are broken after the change. It would be far less painful if there were some transition period during which a request to install a role using the old name (e.g. example-user.example-role) was 'sanitised' to try the new name (example_user.example_role) if the first lookup fails.

The second is that there does not appear to be a recommended way to publish a Github repo such as user/ansible-role-example as user.example on Galaxy. Galaxy V3 recommends using role_name metadata (which triggers the role renames), but as Bharat mentioned, this tag is slated to be deprecated.

[oneswig]

@chouseknecht based on what @markgoddard is saying, would it be possible to expedite a server-side Galaxy fix to lookup roles formulated with either underscores or hyphens? This would help users transition.

[chouseknecht]

@brtknr

The naming changes were made back in June with the release of v3.0. Documentation can be found here.

The intention is to deprecate and ultimately remove the role_name parameter from meta/main.yml. The exact timing has not been decided.

The reason for changing how repositories and roles are named is that we're moving toward support for multiple roles within a repository, as well as support for modules and plugins within a repository. A repository will become a Collection of related content, rather than just a single role.

[brtknr]

So what do you recommend we do in order to minimise disruption on our end? Should we rename all our GitHub repos and remove role_name parameter from meta/main.yml in anticipation of the coming change? We would like clarity on the course of action.

[markgoddard]

Crucially, we would like a path forward that does not involve a breaking change for consumers of our roles. If we have to do work as role producers to make that possible that would be acceptable.

[chouseknecht]

@brtknr

I would say, yes. Remove '-' from repository names, and name repositories the way you want them to appear in Galaxy. And to make life easier in the long run, don't use the role_name setting in meta/main.yml.

[markgoddard]

@chouseknecht thanks for the response. I don't think it is always practical to name git repos the same as the desired role name. Roles often deploy a piece of software, and a user might have a fork or original source of that software in Github. Normally you'd want the role to be named the same as the software, but there would be a name collision in this case. This is exactly why almost every role I've seen has an ansible- or ansible-role- prefix on Github.

Do you have a recommended path to avoid breaking changes for consumers of Galaxy roles, such as the fallback to using a sanitised name on the server side if the first role lookup fails?

[chouseknecht]

@markgoddard

You can keep the ansible- and ansible-role- in the repo name, and use role_name for now to force dropping these prefixes during import. Just know that at some point role_name will be deprecated.

As for the '-', if you would like us to restore the '-' in your existing roles we can. Just know that if and when you delete a role and re-import it, the '-' will again be converted to '_'.

[markgoddard]

Thanks @chouseknecht - that (renaming _ to -) would get us out of a sticky situation for the time being. I trust that if we do not delete the role in Galaxy then imports will not trigger a rename?

Looking to the future, will there come a time when we need to change these names to use underscores? If so, it will be crucial to have a backwards compatible migration path that does not break users and CI/CD pipelines everywhere (not just ours).

IMO without the prefix magic, some solution will be required for setting the name of a role, role_name or otherwise.

[chouseknecht]

@markgoddard

All the repository and content objects (roles) have been renamed with '_' converted to '-'. The import process will leave the '-', unless you delete the role and re-import.

[chouseknecht]

@markgoddard

We have no plans to rename things en masse. At this point, it will be up to you guys to rename/migrate the content as you see fit.

[brtknr]

Many thanks @chouseknecht. All appears to be importing fine including via Travis CI. I just want to raise 2 more minor issues:

• I forgot to remove role_name tag from meta/main.yml for this repo: stackhpc.monasca-agent so it has returned to the underscored state :( would you please kindly put it right again?
• I noticed thatstackhpc.container-clusters is actually supposed to be stackhpc.os-container-clusters based on its GitHub repo name. I believe I may have been using this repo to check how role_name affected the way galaxy applied updates to the role name rendered on Ansible Galaxy and so must have left it in this altered state.

[chouseknecht]

@brtknr

Done.

[brtknr]

Thanks again @chouseknecht.

Do you happen to know why the import for monasca-agent is failing? This is the import log:

INFO Starting import: task_id=282444, repository=stackhpc/ansible-role-monasca-agent
INFO Content search - Looking for file "apb.yml"
INFO Content search - Looking for top level role metadata file
WARNING [role] Found "categories" in metadata. Update the metadata to use "galaxy_tags" rather than categories.
INFO [role] Linting...
ERROR [role] ./library/monasca_agent_plugin.py:9:121: E501 line too long (151 > 120 characters)
ERROR [role] ./library/monasca_agent_plugin.py:32:121: E501 line too long (143 > 120 characters)
ERROR [role] ./library/monasca_agent_plugin.py:52:1: F403 'from ansible.module_utils.basic import *' used; unable to detect undefined names
ERROR [role] ./library/monasca_agent_plugin.py:56:14: F405 'AnsibleModule' may be undefined, or defined from star imports: ansible.module_utils.basic
ERROR [role] ./library/monasca_agent_plugin.py:87:121: E501 line too long (124 > 120 characters)
ERROR [role] ./library/monasca_agent_plugin.py:96:1: E305 expected 2 blank lines after class or function definition, found 1
ERROR Lint failed
INFO [role] Found Content instance: id=22405, content_type="role", name="monasca-agent"
INFO [role] Adding role metadata tags
INFO [role] Adding role platforms
Task "282444" failed: function upper(text[]) does not exist LINE 1: ...form" WHERE UPPER("main_platform"."name"::text) = UPPER(ARRA... ^ HINT: No function matches the given name and argument typ

https://galaxy.ansible.com/my-imports?namespace=stackhpc&repository_name=monasca-agent&selected=282444&page_size=10

[chouseknecht]

@brtknr

Yep. Looks like platforms is malformed in meta/main.yml. You currently have this:

platforms:
- name:
  - Ubuntu
  - Centos
  versions:
  - all
 ```

It should look something like the following:

```yaml
platforms:
- name: Ubuntu
  versions:
    - all
- name: Centos
  versions:
  - all

[brtknr]

Thanks @chouseknecht, that appears to have fixed everything for now.

[luisico]

Late to the "party" (more of a nightmare!), but still wanted to express my opinion about the forced renames. Briefly, a major blow for the community as a whole a) without a logical reason (what does "support for multiple roles within a repository" have to do with dash vs underscore, or what ever users want to use); b) without proper planning (today use role_name, next day don't, or may-be-yes-or-may-be-no).
My 2cts