Instafeed.js: Caching results

[vojtasvoboda]

After a few hours of using Instafeed.js I'm getting HTTP 403 IGApiException (#4): Application request limit reached.

Is there some way how to cache Instagram images fetched over the API, let say for 1 hour?

Thanks for any hint

[benjamin-hull]

Hey @vojtasvoboda - could you just confirm a couple of things, please?

• Are you using the latest version of Instafeed.js?
• Are you using this in a production setting? (eg. with many users all making independent requests from their browsers?)

[vojtasvoboda]

Hi @benjamin-hull I'm using version v2.0.0rc1 in a production setting. Yes, many users all making independent requests.

I think, saving response from the Instagram API to the local storage for a couple of minutes could help a lot.

[stevenschobert]

Hey @vojtasvoboda -- thanks for the feedback. Adding caching is definitely something we've been considering, but haven't implemented yet.

Client-side caching doesn't 100% solve the API limit problem, since a large number of new page visitors (who haven't cached yet) could still exceed the API limits, but it's a nice start.

If you need a solution urgently, you can probably implement the caching yourself using the success callback. If you need more help on how to do that, let us know.

[vojtasvoboda]

Hi @stevenschobert - as I understand the documentation correctly, the API limit is calculated as:

Calls within one hour = 200 * Number of Users

So the absolute number of users is not limiting, but one user loading feed many times per hour is the core of the problem, where the caching could help a lot.

For now, success callback could be used for saving fetched results. But how I can 'inject' results from the cache back to the library? :-) I can save results to the cache, but still want to use rendering from this library, when I'm using some transformations, filters, limiting etc.

[stevenschobert]

Yeah, it's not clear to me what Number of Users means in their docs. I assumed it meant the total number of users of the Facebook App that access token is associated to, but maybe I'm wrong about that.

---

As for injecting the results back into the library, we definitely need to expose that better. Currently, you would need to call the internal methods yourself to process the data and render it.

The main two methods are _processData and _renderData.

You can see an example of how these are used internally here: https://github.com/stevenschobert/instafeed.js/blob/dc304bb3a3d9f488b04d81bf46d8e1ad2f9c62d5/src/instafeed.js#L121-L152

These methods are isolated and don't produce side-effects, so you should be able to "safely" call them to get your HTML string.

You'll have to skip calling run, and inject the HTML into the DOM yourself -- which isn't great. We'll be able to improve that in the future I think.

Other options are:

• temporarily forking/customizing the library to implement the caching yourself
• Cache the entire rendered HTML (instead of just the API data) using the after callback -- and using that instead.

[benjamin-hull]

My reading of Users is 'individual API tokens' - so even with caching instafeed will still be limited to 200 individual users per hour. I'm pretty confident that's the case - in a more 'normal' application setting, each user of your app would be issued their own token, and so they'd each get 200 requests an hour.

The real way to fix that of course is with some shared server-side caching, but that's outside the scope of instafeed.js.

@stevenschobert, it might be useful (and relatively simple) to add a before hook to the options so that people could do a check of their own cache before actually firing the request? (Until we have a built-in cache, anyway)

[benjamin-hull]

Preliminary work on response cache is here: https://github.com/stevenschobert/instafeed.js/tree/656-response-cache. Some notes about the current implementation:

• Uses localStorage/sessionStorage where available
• Currently set to 60 second cache, mainly for testing.
• Not sure if caching should be on or off by default. Off might be better/less confusing for development work, but will mean people won't get the benefit without an extra step.

[benjamin-hull]

I also found some more info in the Facebook app dashboard, under Instagram > Basic Display Rate Limiting, clarifying the limits:

About Basic Display Application-Level Rate Limiting

Rate limiting defines limits on how many API calls can be made within a specified time period. Application-level rate limits apply to calls made using any access token other than a Page access token and ads APIs calls. The total number of calls your app can make per hour is 200 times the number of users. Please note this isn't a per-user limit. Any individual user can make more than 200 calls per hour, as long as the total for all users does not exceed the app maximum.

This is good news - it means you can increase your total request limit by 200 per hour by adding additional Instagram users as testers to your app. You don't need to do anything _with_ those users - simply having them added is enough. This is kind of a weird work around, but if you have a few people on your team you can add, you can increase the request limit by a decent amount.

Combining this with client-side caching should help a lot.

[benjamin-hull]

Bad news on the request limit front, I'm afraid. Basic Display apps use a per-authorized-user request limit which can't be increased, and isn't published for privacy reasons. More info On this Instagram Token Agent issue and in the Facebook docs

[stale[bot]]

This issue has been automatically marked as stale because it hasn't had new comments in the last 3 months. It will be closed if no further activity occurs. If you still need assistance with this issue, or believe it shouldn't be closed, please respond with a new comment to let us know.
Thank you all for your contributions.