Libelektra: How to check if the cache plugin works?

[markus2330]

Steps to Reproduce the Problem

kdb info cache says: "The cache plugin is compiled and enabled on compatible systems by default. No actions are needed to enable it."

kdb mount
#>
kdb mount /large/file.json system/json/test json 

Expected Result

That kdb ls / will be faster the second time. And that kdb gmount cache fails (like it is the case for spec).

Actual Result

time kdb ls / > /dev/null
#> kdb ls / > /dev/null  0,73s user 0,02s system 94% cpu 0,795 total
time kdb ls / > /dev/null
#> kdb ls / > /dev/null  0,82s user 0,01s system 98% cpu 0,840 total
kdb gmount
#> 
kdb gmount cache
#> 
kdb gmount
#> spec
#> cache

System Information

• Elektra Version: master

[mpranj]

Currently, there is no easy way to detect cache hits. Maybe we could add a way of checking for cache hit (adding a meta key to the parent, or enforcing cache by a custom command kdb cache-get).

Another problem here is that the cache is not mounted in the usual fashion (using list). The idea here was to avoid overhead. We can either mount it with list as usual (make it gmount compatible), or add a custom command (kdb cache enable, kdb cache disable).

Another handy thing would be kdb cache clear to remove all cached files.

[markus2330]

Thank you for the answer! Do you have any idea why the parsing of JSON did not speed up in the example above? Can you give an example where the cache benefit is clearly shown?

[mpranj]

Testing it with / can be problematic for many reasons. A cache miss could easily be caused by non-file based storage plugins (#2454), mounted but non-existent config files (resolver says NO_UPDATE, regardless whether it is up to date or the file does not exist) ...

It might just work if you try getting system/json/test.

Another way to test it out is the benchmark I added (./bin/benchmark_kdb).

[markus2330]

It might just work if you try getting system/json/test.

I tried getting several mountpoints (with kdb ls) which are big enough that the cache could pay off. Unfortunately I did not see any improvement with any of them. It also seems like that ~/.cache/elektra is not written to anymore. What exactly does kdb gmount needs to show so that it is present?

The benchmark works but it does not give me any comparison:

./bin/benchmark_kdb
plugin;operation;microseconds
core;kdbOpen;212
core;kdbGet;86
core;kdbSet;44
core;kdbOpen;93
core;kdbGet;57
core;kdbOpen;74
core;kdbGet;26
core;kdbOpen;71
core;kdbGet;25
core;kdbOpen;40
core;kdbGet;55
core;kdbOpen;39
core;kdbGet;24
core;kdbOpen;39
core;kdbGet;24
core;kdbOpen;40
core;kdbGet;24

[mpranj]

Something is definitely off here. The benchmark inserts 40k keys, the first kdbGet is slow and then we should have only cache hits. This is how it looks on my machine:

./bin/benchmark_kdb 
plugin;operation;microseconds
core;kdbOpen;6707
core;kdbGet;758
core;kdbSet;171587
core;kdbOpen;449
core;kdbGet;134177
core;kdbOpen;406
core;kdbGet;12351
core;kdbOpen;356
core;kdbGet;11514
core;kdbOpen;379
core;kdbGet;12184
core;kdbOpen;361
core;kdbGet;11556
core;kdbOpen;377
core;kdbGet;12263
core;kdbOpen;367
core;kdbGet;11444

Note that I have only the default mountpoint. It seems that in your case even the set has failed, or maybe even kdbOpen.

I don't know what's going on here. :confused:

[markus2330]

I think we need some debugging mechanisms for the cache. Maybe like you suggested before: that the cache adds some meta data to the parent key and the kdb command prints it on request.

I think for now you can simply add a warning if the cache does not work (together with a reason why it did not.) Later we might introduce logging/info messages (even less severity of warning, only displayed with -v).

[mpranj]

That makes sense to me too. Up to now I painfully tracked/debugged it using the logger.

Warnings on a cache miss are definitely not usable for production, but info severity would be great for this.

[markus2330]

Will be resolved as discussed.