Badger: How can I estimate the maxmemory ?

[shengery]

Greetings,
Thanks to badger, such an excellent embeded kv store, I am working in a game company, now we are planning to use badger as an in process cache service (like memcached), so as to reduce pressure of the center data server, the procedure is :

1. fetch the data from data server,
2. store into the process's badger
3. fetch from badger
4. write back to data server as needed
   my main purpose is to reduce the multi get requests from data server, the question is how can I set maxmemory of badger or how can I estimate the maxmemory of objects stored in the badger? For I worry about the abuse of memory when I have no way to evaluate it.

Best Regards.

[manishrjain]

You can get the total size of key-values stored in Badger, by doing a key-only iteration, and then use EstimateSize in Item to get the size. Not sure if that's what you're looking for.

[shengery]

Thanks
My purpose is to limit the memory usage when I used badger as a cache layer, the total items maybe vary,
However I hope I can control the maxmemory of the badger by setting some parameters of the Option , if it's not possible I hope there is a way to figure out the memory usage according to the total number of items, key size and value size along with the number of the most frequent accessed items.

[prune998]

It's been a long time ago, but we're using badger as a "read only" DB + our own facade and we're running all that in Kubernetes pods.

It sounds like Badger is using a lot of memory, as much memory as the DB size (which is 5gig for us). Memory size then lower when using the FileIO option instead of the MMAP one, but it still require lots of memory to start.
When I say start, it's just "opening the DB" like kv, err := badger.Open(opt).

So I do have quite the same question as this post :

• how to estimate the needed memory to open a DB file (with no requests) ?

I would extend it with this question :

• how to limit the memory that Badger can use ? It seems to be using "whatever is needed up to the existing memory". (which is a problem for us as containers show the full host memory)

[campoy]

Let's keep it simple and just answer the original question to this thread.
Is there any way to estimate the size in memory given an encoded file?

Assigning the question to @manishrjain since this is something that might be changing with v2.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue was marked as stale and no activity has occurred since then, therefore it will now be closed. Please, reopen if the issue is still relevant.