Shields: Strategy for avoiding rate limiting issues on Heroku

[calebcartwright]

A few of the upstream services Shields integrates with had whitelisted the IP addresses of our OVH badge servers so that Shields would not hit rate limits.

We started running an experiment hosting production on Heroku in response to some availability/performance issues we've had recently (#4878), but now our badge requests are no longer coming from static IP addresses which cause a few rate limiting issues (#4957)

In the short term we've implemented a workaround by proxying the requests through OVH badge servers (#4956) but we're going to need a long term solution.

There's more detailed discussions in various channels on the shields Discord server, but a few potential action items:

• Get a list of upstream services that are whitelisting the OVH Server IP addresses to avoid rate limits for Shields.io
• Investigate alternative approaches to the OVH proxy, such as using Heroku static IP addons, running our own proxy server, and/or working with the upstream providers to get a token Shields can use for authentication that will allow Shields.io to avoid those rate limiting levels

[paulmelnikow]

At the cost of what these services provide, and the simplicity of what needs to be deployed, I'm inclined to run this ourselves on a vps, perhaps with CloudFlare in front of it. I'm not super familiar with how proxying works, or how to cache it privately, but it seems like we could assemble a solution that way.

I do think caching upstream responses is an area where we can achieve some performance growth, though getting this stabilized would be a higher priority.

[paulmelnikow]

Besides Discord, did you identify any other affected services?

[chris48s]

I think we said https://hex.pm/ might have whitelisted our IPs at some point, but we've not had any reports that their badges have stopped working since the move

[Daniel15]

working with the upstream providers to get a token Shields can use for authentication that will allow Shields.io to avoid those rate limiting levels

This seems the most sustainable. IP whitelisting isn't really sustainable long-term as it makes it more difficult to move to another server provider, add IPv6 in the future, etc. It's better to have some way to authenticate requests are coming from Shields.io, whether that be JWT tokens, OAuth'd requests, whatever.

[PyvesB]

We've successfully implemented a token-based approach for Discord, and Hex.pm turned out not to be a problem in the end. Let's discuss decommissioning the old OVH servers during next Operations meeting!