Mybb: Cache Improvements

[kawaii]

So I've been thinking about this for a while. Originally I just wanted to raise an issue to add a Redis cache driver to MyBB 1.9, then I noted smaller unoptimised parts of the codebase like loading the entire bans cache on every page load, which sparked this lengthy debate on how useful caches in MyBB are at all. We already know if the user is a member of any 'banned' groups, so why not simply do an indexed search on the bans table to pull the rest of the information if so? Larger forums with hundreds if not thousands of bans lament this functionality, it's completely wasteful on resources.

In general, MyBB's memory cache support (memcached, ACP and the rest) are rendered more or less useless when you realise you only save one or two queries per page. For example, the delayed update thread views system could just increment a count in memory which then periodically gets flushed to the database, rather than the current system of inserting rows into a table (and since it's in memory, you could also display an accurate view count, which does also eliminate the 'delayed' downside to the option). As of now, updates to the datacache always need a write to the database, which makes it less attractive as a temporary store for batching/delaying writes, something that an in-memory cache is actually quite good for.

@mybb/developers please leave your comments and suggestions here.

[euantorano]

I originally planned to integrate Laravel’s cache package as part of 1.9. It was just going to be a straight swap at first without too much of a logical change around it, but this could just be a first step with a plan for more radical changes as time goes on.

On 31 May 2019, at 21:35, Kane Valentine [e-mail] wrote:

So I've been thinking about this for a while. Originally I just wanted to raise an issue to add a Redis cache driver to MyBB 1.9, then I noted smaller unoptimised parts of the codebase like loading the entire bans cache on every page load, which sparked this lengthy debate on how useful caches in MyBB are at all. We already know if the user is a member of any 'banned' groups, so why not simply do an indexed search on the bans table to pull the rest of the information if so? Larger forums with hundreds if not thousands of bans lament this functionality, it's completely wasteful on resources.

In general, MyBB's memory cache support (memcached, ACP and the rest) are rendered more or less useless when you realise you only save one or two queries per page. For example, the delayed update thread views system could just increment a count in memory which then periodically gets flushed to the database, rather than the current system of inserting rows into a table (and since it's in memory, you could also display an accurate view count, which does also eliminate the 'delayed' downside to the option). As of now, updates to the datacache always need a write to the database, which makes it less attractive as a temporary store for batching/delaying writes, something that an in-memory cache is actually quite good for.

@mybb/developers please leave your comments and suggestions here.

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[Azareal]

I agree with that.

I will add that you can probably cut a fair bit of data going between the forum and the database, simply by loading templates off the disk instead of the database, if you're not doing that already.

The main issue with the bans cache is that the larger it is, the longer it'll take to be de-serialized from
a string and into memory and each and every request will end up with their own copy of the entire bans table in memory.

If you really want a speed hack, you can put the expiry time on the users table, you can use -1 to represent not banned and check if the current time is over the expiry, but I'm sceptical of the benefits of this.

If I recall, the forum permissions cache, which I've talked about before, is a similar beast.
Perhaps, instead of loading the entire thing at once, a smaller and more relevant portion could be loaded for a specific request.

[floraiNx-zz]

Bumping this. I'm sitting at 42.238 banned accounts, most of them permanent. Here is how my query looks like from mybb_datacache.

[kanadaj]

Just to add to this issue with our experience from a few years ago; our use case required us creating a very large number of subforums. Since the entire forums table is serialized into a string, this results in truly massive cache sizes; at some point we were looking at ~50MB of data to deserialize from forums and forum permission on every pageview. On PHP 5.6 this also resulted in ridiculous memory leaks but I suspect that has probably been fixed by now.

Removing descriptions from every forum reduced the cache massively and only came back around the 5000 forums mark, at which point we were looking at intermittent page errors all over the place.