Pokeapi: Should we still use Cloudflare, and if so, is our caching configuration optimal?

[tdmalone]

In #353:

Are we really sure to continue to use cloudflare? In my opinion it caused too many cors problems.

According to #352 though, we _may_ get close to Netlify’s bandwidth limits.

@naramsim, what were the CORS issues with Cloudflare? Do you know if they were due to configuration, or whether they’re problems in Cloudflare’s approach?

[Naramsim]

This is the thread: https://github.com/PokeAPI/pokeapi/issues/289

Moreover, across these years, this API hasn't been stable. By stable I mean the response time. It could vary from 0.1 seconds to more than 10 seconds. If we go for static files the API will be much more stable and probably faster than ever. This will attract more users, and then more bandwidth.

[sargunv]

Yep, the static version responds effectively instantly to every request. Let's see how bandwidth usage is once we've moved the domain, and discuss cloudflare after we've got real data.

[cmmartti]

In api-data#7:

Ideally we'd have all our API requests run through a caching and rate limiting proxy anyway (does Cloudflare do that?).

It looks like Cloudfare can do rate limiting fairly easily.

Notably:

Cached resources are never rate-limited, even if they match a rule.

Once a client exceeds the threshold specified in the time period, the rule will trigger. The periods available, by plan, are:

• 1 second or 1 minute (Free and Pro)
• 1 second, 1 minute, or 10 minutes (Business)
• Any period between 1 second and 1 hour (Enterprise)

If the docs page is accurate, the current rate limit is 300 requests per resource per IP address per day (so a single IP address can call the "bulbasaur" resource 300 times a day). Creating a rate limit like this in Cloudfare is (a) not possible, because of the max time period of 1 hour, and (b) not necessary, because the resource will be cached anyway.

This raises the question of whether we need rate limiting at all. In my opinion, we should set a reasonably high limit of _x_ API requests per minute (I don't know what that number would be), and only lower the limit if we find that certain clients are abusing the service.

[sargunv]

How does CloudFlare's caching work? Will it be able to cache responses to requests with specific GET queries? If so, it should work well to reduce our lambda function usage with resource lists. I'd still recommend some generous rate limiting (unreachable by normal use), since a malicious client could use up our lambda function quota by requesting various combinations of offset and limit on a sufficiently large resource list.

[cmmartti]

It looks like CloudFlare does not cache JSON/HTML by default, but it can be set to cache everything by creating a wildcard page rule like pokeapi.co*.

As far as requests with GET query strings, it looks like it does cache those, but every permutation is cached separately unless query string sorting is enabled, which requires an Enterprise plan.

[sargunv]

That sounds like it should work for us. Anyone want to test it out with our static API and see how it goes?

[tdmalone]

Update: given we now have access to PokeAPI's Cloudflare account but not yet the domain itself; and given we're likely to be hosting on Firebase now rather than Netlify and I don't _think_ Firebase does DNS hosting directly, I think it makes the most sense to continue using Cloudflare.

Leaving this issue open, however, as a pointer to look into/tweak the caching setup.

[cmmartti]

I don't know how I missed this before, but it looks like Cloudflare charges $0.05 per 10K requests for rate limiting after the first 10K (only unblocked requests are counted). Paul said PokeAPI gets 30K-50K requests per day. Assuming 50K requests per day, that would be roughly $7–$8 per month for rate limiting.

I did some testing on my own domain, and rate limiting seems to work okay. However, I came across this comment on Stack Overflow by a Firebase engineer:

Please _don't_ put Cloudflare in front of your Firebase Hosting site. We already serve everything from a CDN, so putting another CDN in front of it will actually make it slower! – Michael Bleigh Oct 17 '17 at 1:07

Firebase Hosting doesn't offer any rate limiting itself.

As far as Firebase Hosting costs goes, it's $0.15/GB transferred after the first 10 GB each month, and cloud functions is $0.40/million invocations after the first 125K each month. If we're serving as many as 1.5 million requests each month, that's definitely over 10 GB. If we assume each request is served 0.2 MB, and we get 50K x 30 requests per month, that's 300 GB, which is $45.

If we used Cloudflare with rate limiting and an aggressive cache setting of one day, that might be enough to stay in the free Firebase quota.

A while back, @sargunv on Slack said did a cost breakdown (I'm not sure what the difference between "Hosting - GB transferred" and "Cloud Functions - Outbound networking" is):

With cloud functions we get 2M invocations, 200K CPU seconds, and 5GB networking egress per month for free. Above the free quota, invocations appear to be $0.40 per 1M invocations, CPU $1 per 100K seconds, and network $0.12 per GB.

Also memory: 400K GB-seconds for free + $0.25/100K GB-seconds
The static files (used by the cloud functions) are 1GB stored free with 10GB transferred free + $0.15/GB transfer. Not sure if that includes transfer within their network but we can assume it does

Function network egress seems to be the limiting factor here

[sargunv]

We should enable caching for cloud functions to take advantage of Google's CDN for those requests.

[cmmartti]

Good point. We can do that by setting the Cache-Control response header in the cloud function, as described here.

[cmmartti]

So what's the consensus? Do we need Cloudflare? Rate limiting might be a good idea, and we can't do that without Cloudflare. Also, if we want to stay within Firebase's free quota, Cloudflare's cache may or may not handle that if we set a long cache period (one or more days).

[tdmalone]

We'll need to use Cloudflare for DNS either way, and we can turn on/off Cloudflare's CDN et al functionality at will (by just flicking the orange cloud in the console 😄).

So I think possibly to avoid this being any sort of a blocker: let's say we'll leave it switched on for now? But we can easily review and tweak if we need to.

Note that we currently aren't using rate limiting, but we do have a few access rules in place for setting up JavaScript-based challenges for requests from some countries.

(Unfortunately our current Cloudflare account is part of Paul's account with other domains in it, so access is limited, but both @Naramsim and I can make changes when needed - in particular, flicking the DNS over to Firebase as soon as we're good to go there).

[Naramsim]

Yes, we should use it to cache most of the requests.