Galaxy: Add notification support for travis-ci.com

[haxorof]

Bug Report

SUMMARY

Projects in Travis CI has been migrated from travis-ci.org to travis-ci.com and notifications does no longer work but instead HTTP 400 is returned from the notification API.

I have again went through the instructions listed in the documentation: https://docs.ansible.com/ansible/latest/reference_appendices/galaxy.html#travis-integrations

STEPS TO REPRODUCE

Do not know a good way to reproduce it more than doing a commit in github and wait for Travis CI to send a webhook notification.

EXPECTED RESULTS

Import triggered in Ansible Galaxy

ACTUAL RESULTS

No trigger seen in Ansible Galaxy and Travis CI got 400 error.

Got a log from Travis CI support to just show that the API returns 400. They also say they seen these kind of errors on other projects since at least 5th of June:

Jun 07 09:21:19 UTC task=webhook status=error build=75601644 url=https://galaxy.ansible.com/api/v1/notifications/ error_code=400 message="detail" 

Travis CI webhook payload can be seen here:
https://github.com/travis-ci/travis-tasks/blob/b70c47a4935f3c3ecb3d096e5ad39c56d7d6f465/lib/travis/addons/webhook/payload.rb

[webknjaz]

Galaxy probably needs to implement an additional way to separate .com from .org installations. Other services like codecov.io faced the same issue a while back. Badges service shields.io has recently implemented a separate URL for .com badge. Please also note that the projects might have existed in both .org and .com simultaneously at some point.

[haxorof]

@webknjaz : So have I understood you correctly that basically the problem I'm experiencing right now will not be resolved until something is done in Galaxy?

As you also mention my projects actually did exist in both places (travis-ci.org and travis-ci.com) at some point in time for a day or two.

[webknjaz]

Existing in both places is an artifact of .com being for paid customers only earlier. So one could have a project in both places before they decided to merge.

[webknjaz]

And yes, it looks like galaxy should expose an extra API endpoint.

[chouseknecht]

Galaxy was built with the assumption that CI would run on the public Travis site, travis-ci.org. We check the signature of the inbound request received from Travis using the travis-ci.org public key, and so my guess is that requests received from travis-ci.com are signed with a different public key.

Here's where we do the checking, btw:

https://github.com/ansible/galaxy/blob/develop/galaxy/api/views.py#L979

Marking this an enhancement. I think you're the first person to ask about this, so not sure how high a priority we'll give it.

[haxorof]

@chouseknecht : Thanks! I just wanted to make you aware of this because Travis CI are moving open source projects from travis-ci.org to travis-ci.com (https://docs.travis-ci.com/user/open-source-on-travis-ci-com/) which started in May.

I will manage in the meantime without notification.

Cheers!

[webknjaz]

@chouseknecht

Now it's a manual opt-in and new repos only, but they'll start gradually migrating all repos in the end of Q2 2018. I've got one whole organization I maintain fully migrated about a month ago.

The doc https://docs.travis-ci.com/user/notifications#Verifying-Webhook-requests says:

Obtain the public key corresponding to the private key that signed the payload. This is available at the /config endpoint’s config.notifications.webhook.public_key on the relevant API server. (e.g., https://api.travis-ci.com/config)

So you should validate against both keys in https://api.travis-ci.com/config and https://api.travis-ci.org/config to identify where it comes from.

This method https://github.com/ansible/galaxy/blob/develop/galaxy/api/views.py#L996 should grab two keys and it should probably be upgraded to looping against a list of trusted keys.

See this note: https://gist.github.com/andrewgross/8ba32af80ecccb894b82774782e7dcd4#file-travis_webhook_checker-py-L22-L26

[webknjaz]

P.S. Since all new repos go to .com it might make sense to prioritize this.

[chouseknecht]

@haxorof @webknjaz

Fix pushed to https://galaxy-qa.ansible.com. Our QA server should now accept requests from travis-ci.com.

Can one of you guys test by updating the notification webhook in your .travis.yml to call https://galaxy-qa.ansible.com/api/v1/notifications/

[webknjaz]

Sorry, I don't have a project to try it on. Let's wait for @haxorof :)

[haxorof]

@chouseknecht Sorry not answering earlier but I will see if I can test this some time today or tomorrow.

[haxorof]

@chouseknecht Now I have tested it two times against Galaxy QA and it works like a charm. Good work!