Libelektra: Dependency issues/segfaults related to cache plugin

[kodebach]

After updating the master branch I had major problems with kdb. Both of them are related to the cache plugin and both are missing dependencies in CMake.

1. The cache plugin is 100% required for kdb to work. Even a simple call to kdb without arguments results in a segfault, if the cache plugin is not found. Therefore the cache plugin should be a dependency of kdb in CMake. Here is the valgrind output for the segfault. Also, the segfault shouldn't happen, even if cache isn't found, ideally kdb would just run without a cache, but a meaningful error message is the least that I expect.
2. Once the cache plugin is found. The mmapstorage plugin caused errors. I assume it is a dependency of cache (not mentioned in the README). Therefore it should be a CMake dependency of cache. I had an old version of mmapstorage so I got some weird error message. If mmapstorage isn't present, I get a segfault very similar to the one above (stacktrace is the same).

Maybe we should add a CI job that only executes make kdb instead of make all to ensure kdb can be built and run without having to build all of Elektra...

[mpranj]

Thank you for reporting!

Both problems were already encountered on the build server, were fixed, and should make no problems.

ad 1.) the cache is only mounted if it is compiled/available

ad 2.) the cache is only compiled if resolver and mmapstorage are available

The cache definitely needs mmapstorage from the same revision. I do not really understand how you can have an old version of mmapstorage.

Both problems are definitely not by design, so I will see if I can reproduce them and make it less error prone.

[kodebach]

the cache is only mounted if it is compiled/available

Then there has to be a bug somewhere. If I remove libelektra-cache.so and then run kdb, I get the segfault until I restore the file (e.g. by rebuilding it).

the cache is only compiled if resolver and mmapstorage are available

Yes that is true, but it doesn't declare a dependency (with add_dependencies or the DEPENDS arg of add_plugin) on either of them, so they aren't built automatically.

The cache definitely needs mmapstorage from the same revision.

If this is a requirement even after release, you should add some kind of version check. Generally Elektra tries to be binary compatible AFAIK.

I do not really understand how you can have an old version of mmapstorage.

Its quite easy:

• Remove the lib folder from your CMake build directory
• Checkout an old commit pre cache plugin (e.g. a808874567b2c0c823f7619fa72078d89802bca1)
• Run make all
• Checkout master (I used 152d5a985b20afd509dc26b93563a7bc9564ceaf to be precise)
• Run make kdb

There is no CMake dependency between kdb and elektra-cache or elektra-mmapstorage (or between elektra-cache and elektra-mmapstorage), so neither cache nor mmapstorage will be built which somehow leads to a segfault. Normally a missing plugin leads to some error mentioning dlopen or libsomething.so, but this is not the case here.

Once you run make elektra-cache and make elektra-mmapstorage everything works as expected.

The segfault shouldn't occur in any case, but the problem of cache/mmapstorage not being present can easily be fixed by adding the right add_dependencies to CMake. See also #2555 for a similar issue.

[mpranj]

Thank you for the details, this is indeed helpful!

If I remove libelektra-cache.so and then [...]

Ah yes, the check is static and does not work if the file is removed.

Yes that is true, but it doesn't declare a dependency [...]

Thank you for the hint, I overlooked this.

If this is a requirement even after release [...]

I'll keep it in mind but there should be no breaking changes after release.

[markus2330]

ideally kdb would just run without a cache, but a meaningful error message is the least that I expect.

I fully agree here. Shouldn't this be automatically the case? (As long as cache is not mounted globally, it should not do anything, or?)

Generally Elektra tries to be binary compatible AFAIK.

Yes, Elektra tries to be binary compatible.

Maybe we should add a CI job that only executes make kdb instead of make all to ensure kdb can be built and run without having to build all of Elektra...

@kodebach: if this is part of your workflow it totally makes sense that you add a build job that checks this.

The segfault shouldn't occur in any case

Yes, there should definitely also a run-time check if the dlopen worked.

[mpranj]

Currently it is mounted globally by default. I thought this was the goal in the long run.

[markus2330]

Yes, globally by default is perfectly fine. If it is already the case, then please fix the segfaults asap (with run-time checks).

[markus2330]

@kodebach it would have made sense to make two separate issues here.

The segfault one is urgent but the dependency one is not.

[mpranj]

I'm almost done with it :wink: