Parse-server: Cache grows when not using enableSingleSchemaCache

[flovilmart]

Issue Description

It has been observer that the new caching mechanism (LRUCache) would not prune proactively it's cache after a certain amount of time, leading to memory growth and explosion.

Steps to reproduce

• Startup a parse-server, without enableSingleSchemaCache
• use heapdump through require('heapdump') in your startup script
• notice that the memory keeps growing after each request.

Expected Results

Memory usage should somewhat remain constant, unused data should not be kept indefinitely.

Environment Setup

• Server

  • parse-server version (Be specific! Don't say 'latest'.) : >= 2.5.1
  • Operating System: n/a
  • Hardware: n/a
  • Localhost or remote server? local

• Database

  • MongoDB version: n/a
  • Storage engine: n/a
  • Hardware: n/a
  • Localhost or remote server? n/a
    

    Logs/Trace

See discussion over: https://github.com/parse-community/parse-server/issues/4235

Suggested resolution:

1- make singleSChemaCache default (this should be pretty harmless)
2- actively cleanup upon response end the schema cache used for the request
3- actively prune LRU cache dropping older values periodically

[iamvajid]

@flovilmart any updates on this issue ? Still looking for someone to take this up ? The label good first issue is tempting me if you can give some courage

[flovilmart]

Yes that should be an easy one, the suggested resolution should be solid at the moment, and it will give you a good insight on the underskirts of parse-server :)

If you run into any issue, just let me know!

[cyrixmorten]

Can confirm that setting enableSingleSchemaCache: true effectively stops the memory consumption from growing.

Exceeded my memory quota on Heroku this morning and after setting enableSingleSchemaCache to true it has been steady at ~100mb

[montymxb]

Yeah we need to address this. I took a look at some of the code at one point. Haven't confirmed it but it does just seem to be a bad case of not cleaning up the multitude of schema caches that accumulate over time. Interesting code being here in SchemaCache.js. Over time with random prefix ids I imagine you could fill up memory no sweat. One way would be to track formerly used caches (particularly stale?) and flush those after a certain count. Maybe a cache of existing cache prefixes that still exist so we can keep track of them?

[flovilmart]

The issue is that:
1- we don’t cleanup the cache / request after the request is over
2- The LRUCache don’t actively prune

We could manually prune the cache just after each requests or periodically, and/or when the request is over; remove the key associated in the cache with it.

[NicksonYap]

Any fix for this so far?
Does enableSingleSchemaCache: true affect anything?

[flovilmart]

It reduces the cache sizes for sure

[NicksonYap]

I suppose a reduced cache size = reduced performance?

[flovilmart]

Nope increased performance. But may yield issues if multiple Schema mutations are run concurrently

[JacobJT]

The _Schema query has been in our slow queries tab on mLab for a while. Will setting this affect the number of _Schema queries, or only the memory due to caching the results?

Edit - I found this in Definitions.js

  "enableSingleSchemaCache": {
    "env": "PARSE_SERVER_ENABLE_SINGLE_SCHEMA_CACHE",
    "help": "Use a single schema cache shared across requests. Reduces number of queries made to _SCHEMA. Defaults to false, i.e. unique schema cache per request.",
    "action": parsers.booleanParser,
    "default": false
  },

So it should also reduce the number of queries run.

[flovilmart]

Yes that’s what it’s meant for

[fsufyan]

I can confirm that setting the enableSingleSchemaCache to true fixes the memory issue and stabilize the RAM usage between 90-150 MB. It was hitting the 440 MB easily before.
Check the screenshot and notice the highlighted area.

Parse 2.7.1
Ubuntu 16.04

[dplewis]

Closing via https://github.com/parse-community/parse-server/pull/5612