Galaxy: Installing roles with dashes in the name is broken

Created on 1 Jul 2018 · 33Comments · Source: ansible/galaxy

Bug Report

SUMMARY

I used to be able to install roles with dashes in the name, e.g. elliotweiser.osx-command-line-tools. I can no longer do so. It appears that underscores are required. _This breaks roles and playbooks that pull in such roles as dependencies._

Example: https://travis-ci.org/geerlingguy/mac-dev-playbook/builds/398684539

STEPS TO REPRODUCE

pip install ansible # occurs w/ 2.6 and 2.5
ansible-galaxy install elliotweiser.osx-command-line-tools

EXPECTED RESULTS

The role should be successfully downloaded.

ACTUAL RESULTS

$ ansible-galaxy install elliotweiser.osx-command-line-tools
- downloading role 'osx-command-line-tools', owned by elliotweiser
 [WARNING]: - elliotweiser.osx-command-line-tools was NOT installed successfully: - sorry, elliotweiser.osx-command-line-tools was not
found on https://galaxy.ansible.com.

ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.

It appears the role can only be installed when using underscores instead of dashes:

$ ansible-galaxy install elliotweiser.osx_command_line_tools
- downloading role 'osx_command_line_tools', owned by elliotweiser
- downloading role from https://github.com/elliotweiser/ansible-osx-command-line-tools/archive/2.1.0.tar.gz
- extracting elliotweiser.osx_command_line_tools to /Users/elliotweiser/git/github/elliotweiser/elliotweiser.osx_command_line_tools
- elliotweiser.osx_command_line_tools (2.1.0) was installed successfully

I'm not sure if this was change in behavior was intentional, but it may have been an oversight.

arebackend priorithigh typbug

Source

elliotweiser

Most helpful comment

Some additional irony...

This URL looks good :+1: https://galaxy.ansible.com/elliotweiser/osx-command-line-tools

This one, not so much... https://galaxy.ansible.com/elliotweiser/osx_command_line_tools

elliotweiser on 1 Jul 2018

😄3 😕1

All 33 comments

Some additional irony...

This URL looks good :+1: https://galaxy.ansible.com/elliotweiser/osx-command-line-tools

This one, not so much... https://galaxy.ansible.com/elliotweiser/osx_command_line_tools

elliotweiser on 1 Jul 2018

😄3 😕1

Related to #776

chouseknecht on 1 Jul 2018

@chouseknecht Thanks for looking into these.

elliotweiser on 1 Jul 2018

In local development, I am able to see in the Debug logging that a new Content record is being created when one clearly should have already existed:

2018-07-01 03:33:13,502 INFO tasks: Starting task: 256196
[2018-07-01 03:33:13,502: INFO/Worker-3] Starting task: 256196
2018-07-01 03:33:15,925 DEBUG base: name: osx-command-line-tools
[2018-07-01 03:33:15,925: DEBUG/Worker-3] name: osx-command-line-tools
2018-07-01 03:33:15,930 DEBUG base: original_name: ansible-osx-command-line-tools
[2018-07-01 03:33:15,930: DEBUG/Worker-3] original_name: ansible-osx-command-line-tools
2018-07-01 03:33:15,953 DEBUG base: Created new Content instance: id=26513, content_type="role", name="None"
[2018-07-01 03:33:15,953: DEBUG/Worker-3] Created new Content instance: id=26513, content_type="role", name="None"

Restored my local DB, and here's the Content record before performing an import

  id   |          name          |     original_name      |            created

-------+------------------------+------------------------+-------------------------------
 14481 | osx-command-line-tools | osx-command-line-tools | 2016-12-24 22:14:19.671032+00

The issue is original_name. In the DB, prior to import, it has a value of 'osx-command-line-tools'. The import process searches for a name of 'ansible-osx-command-line-tools', using repo name as the default value. No match, and so I guess it tries to create a row, but strangely ends up replacing the one that's already there.

Here's the same record after performing the import:

id   |          name          |         original_name          |            created
-------+------------------------+--------------------------------+-------------------------------
26505 | osx_command_line_tools | ansible-osx-command-line-tools | 2018-07-01 08
:11:09.431027+00

Notice the ID and Created values have changed. We actually recreated the record completely.

chouseknecht on 1 Jul 2018

I wonder if this explains why my role :point_up: received a new ID after the 3.0.0 upgrade.

https://github.com/elliotweiser/ansible-osx-command-line-tools/commit/4db0cef508ca6822c2a29fac86c0a98ff7241619

elliotweiser on 1 Jul 2018

@elliotweiser

Fixed by #784. You can test your role at https://galaxy-qa.ansible.com - our QA site, where your role is still named 'osx-command-line-tools'

The patch will be deployed to production tomorrow morning. Let us know, if you want to rename your role in production.

chouseknecht on 2 Jul 2018

👍2

Confirmed:

$ ansible-galaxy install elliotweiser.osx-command-line-tools -s https://galaxy-qa.ansible.com -p roles
- downloading role 'osx-command-line-tools', owned by elliotweiser
- downloading role from https://github.com/elliotweiser/ansible-osx-command-line-tools/archive/2.1.0.tar.gz
- extracting elliotweiser.osx-command-line-tools to roles/elliotweiser.osx-command-line-tools
- elliotweiser.osx-command-line-tools was installed successfully

Thanks for the offer, but I'd rather keep the original name, i.e. osx-command-line-tools (w/ dashes). I can revert the changes I made in response as needed.

elliotweiser on 2 Jul 2018

@cutwater

After deploying 3.0.1 tomorrow morning, please update the Content record for elliotweiser.osx-command-line-tools, replacing '_' with '-' in name.

chouseknecht on 2 Jul 2018

Thanks for pushing the latest release.

I'm still seeing an issue:

INFO Starting import: task_id=259026, repository=robertdebock/ansible-role-python-pip
WARNING [role] 'py-pip' is not a valid tag. Tags must container lowercase letters and digits only. Skipping.
INFO [role] Linting...
INFO [role] Linting OK.
Task "259026" failed: duplicate key value violates unique constraint "main_content_namespace_id_repository__d7de6a83_uniq" DETAIL: Key (namespace_id, repository_id, name, content_type_id)=(4003, 53998, python_pip, 11) already exists.

robertdebock on 3 Jul 2018

@robertdebock

Think it was a timing issue. Re-imported robertdebock/ansible-role-python-pip, and it succeeded.

Let us know, if you want your roles restored to '-'. I see some of them had '-' converted to '_'.

chouseknecht on 3 Jul 2018

@elliotweiser osx-command-line-tools is fixed.

cutwater on 3 Jul 2018

👍1

Looks like repo names are still getting renamed. Pushing #806 to QA for testing.

chouseknecht on 3 Jul 2018

This is very bad. User names were also changed with no notice whatsoever. See also #779

This has broken a bunch of role installs from requirements files and more importantly broken trust with ansible galaxy.

As a user who wants to install a role that used to work fine, how do you think I would react to the following message (using my own insignificant role as an example)?

[WARNING]: - stefano-m.iptables was NOT installed successfully: - sorry, stefano-m.iptables was not found on https://galaxy.ansible.com.

At this point I would expect that the hyphenated names are restored and both the hyphenated and underscored namespaces and packages are linked to the same user.

@chouseknecht I appreciate that you are trying hard to solve the issue. Thanks for working on this.

stefano-m on 3 Jul 2018

@chouseknecht thanks, I'd like to have these roles restored with a "-" instead of "_":

ansible-role-digitalocean-agent
ansible-role-python-pip
ansible-role-sudo-pair

It seems this is something that I can't do right? Otherwise, I'd like to be able to do this myself.

robertdebock on 3 Jul 2018

@robertdebock
Before we do that, remove 'role_name' from the meta/main.yml file of each role. Otherwise, it'll just translate '-' to '_' again on the next import.

chouseknecht on 3 Jul 2018

@elliotweiser

Same as mentioned above, if you put 'role_name' in your metadata, please remove it, and then we can reset the role and repo names, and it should be good.

chouseknecht on 3 Jul 2018

@lae

Same as mentioned above, if you put 'role_name' in your metadata, please remove it, and then we can reset the role and repo names, and it should be good.

chouseknecht on 3 Jul 2018

@chouseknecht reverted

lae on 4 Jul 2018

@lae

Your travis-lxc roles is fixed now. As a test, I re-imported after the change, and both the repo and content name remain as 'travis-lxc'.

chouseknecht on 4 Jul 2018

👍1

I've renamed my roles to use underscores instead of dashes. Thanks for your support @chouseknecht

robertdebock on 5 Jul 2018

@chouseknecht could you please fix andrewrothstein.alpine-glibc-shim and andrewrothstein.java-oracle as well?

spk83 on 5 Jul 2018

@spk83 done!

chouseknecht on 6 Jul 2018

I have the same issue with:

cloudalchemy.node-exporter
cloudalchemy.mysqld-exporter
cloudalchemy.snmp-exporter
cloudalchemy.blackbox-exporter

None of above have role_name in metadata. Can you reimport it with - instead of _?

paulfantom on 7 Jul 2018

@paulfantom

Updated all except cloudalchemy.mysqld-exporter. Don't see that one in Galaxy. Import it, and I'll update it.

chouseknecht on 7 Jul 2018

Sorry, mistake on my side. I have imported that role, it is listed here: https://galaxy.ansible.com/cloudalchemy/ansible_mysqld_exporter with a name ("ansible_mysqld_exporter") which I cannot change, it should be listed as mysql-exporter.

BTW. What will be the mechanism for role naming in future roles? Using role_name in meta/main.yml? Previously it was taking repo name and stripped ansible- or ansible-repo- and voila, but it seems that this behavior changed after 3.0.

paulfantom on 7 Jul 2018

@paulfantom

That last one is fixed. It's now 'mysqld-exporter'.

Going forward, anything new will have '-' converted to '_'.

If you want to alter the name (e.g., remove 'ansible-'), then use the role_name attribute to set the name.

Deleting a role and re-importing, or setting the role_name value, will update the name in the Galaxy database, and cause '-' to be converted to '_'.

chouseknecht on 7 Jul 2018

👎3 👍2

Great feature. Congrats.

...

jnogol on 11 Jul 2018

🎉1

@chouseknecht - Found another one that I missed earlier: https://galaxy.ansible.com/geerlingguy/ssh-chroot-jail — can it be reverted to use - instead of _?

geerlingguy on 18 Jul 2018

@geerlingguy

https://galaxy.ansible.com/geerlingguy/ssh-chroot-jail is fixed.

Sorry for the delay. Didn't see your comment until now.

chouseknecht on 20 Jul 2018

Closing, as I think we've renamed all the roles listed on this issue.

If you find more roles that you want renamed from '_' to '-', please open a new issue, so that we don't miss it.

chouseknecht on 20 Jul 2018

👍1

Not sure if this one is a similar case:
https://galaxy.ansible.com/mattfinlayson/ansible-consul

danielkucera on 19 Nov 2018

What about this?

"Changes to APIs must always be deployed as fully backwards compatible upgrades"

Endpoint stability

dortegau on 8 Jan 2019

👍2

@chouseknecht can u fix the mapping of my roles below into the galaxy?
andrewrothstein.ansible-openjdk -> andrewrothstein.openjdk
andrewrothstein.ansible_restic -> andrewrothstein.restic
thanks!