Libelektra: spec-mount rest-backend-config does not work any more

[BernhardDenner]

Describe what you wanted to do

prepare the rest-backend configuration with kdb mount-rest-backend-config, which involves

kdb mount rest-backend-spec.ini spec/sw/elektra/restbackend/#0/ ni
kdb spec-mount /sw/elektra/restbackend/#0/ ni

This leads to

Got mountpoint from spec/sw/elektra/restbackend/#0/current with 33 nodes, configfile:  and mountpoint: /sw/elektra/restbackend/#0/current/backend/api/description/html
The command kdb spec-mount terminated unsuccessfully with the info:
The path you entered is invalid.
Try to add another path instead.

Filenames are typically (depending on
resolver) not allowed to contain '..'.

For more information see:
 kdb info <your resolver>
Please report the issue at https://issues.libelektra.org/

Describe what you expected

rest-backend-config.ini mounted according to defined specification.

Describe what actually happened

The command kdb spec-mount /sw/elektra/restbackend/#0/ ni gives the above error message.

Steps to reproduce:

# in case you do not have rest-backend compiled
cp <libelektra-src>/src/tools/rest-backend/config-specifiction.ini.in /usr/local/share/elektra/specification/rest-backend-config.ini
# manually substitute cmake vars
vim /usr/local/share/elektra/specification/rest-backend-config.ini
kdb mount rest-backend-config.ini spec/sw/elektra/restbackend/#0 ni
kdb spec-mount /sw/elektra/restbackend/#0

The elektra version GIT sha1: e5624dd2ddb1a9ee09e467a9c65f40f1cea76925 (master from end of May) works as expected.

System Information

• Elektra Version: master
• verified on Debian 9 and Ubuntu 16.04

[BernhardDenner]

Some debugging I did:

The comand kdb ls spec/sw/elektra/restbackend gives

with the current master build (spec-mount not working)

/sw/elektra/restbackend/#0/current/backend/api/description/html
...
spec/sw/elektra/restbackend/#0/current
...

while the old build (from mentioned GIT hash - spec-mount working)

spec/sw/elektra/restbackend/#0/current
...

So, now the kdb ls contains cascading keys too.
I've done a short verification if this triggers the bug with the help of the Ruby bindings

require 'kdbtools'

db = Kdb.open
ks = Kdb::KeySet.new
db.get ks, 'spec/sw/elektra/restbackend/#0'
ks.pretty_print
spr = Kdbtools::SpecReader.new
spr.read_specification ks
m = spr.get_backends
b = m.first[1]
b.nodes             # => 33
b.get_config_file   # => ''
b.get_mountpoint    # => /sw/elektra/restbackend/#0/current/backend/api/description/html

# now create a KeySet without cascading keys
ks_spec = Kdb::KeySet.new(ks.find_all {|x| x.name =~ /^spec/})
spr = Kdbtools::SpecReader.new
spr.read_specification ks_spec
m = spr.get_backends
b = m.first[1]
# those are the values reported by a working kdb spec-mount -v ...
b.nodes             # => 45
b.get_config_file   # => 'rest-backend.ini'
b.get_mountpoint    # => /sw/elektra/restbackend/#0/current

[markus2330]

Hi, I see several problems in your commands, the third problem is the most likely the actual cause:

1. You use config-specification.ini from the src dir, but you should use the one from the build dir. You might have wrongly substituted values. Correct copy command for me is: cp ~build/src/tools/rest-backend/config-specification.ini /usr/local/share/elektra/specification/rest-backend-config.ini
2. The path you entered is non-default: It is /usr/local/share/elektra/specification/ (without s at the end, see above).
3. The specification file needs to be mounted with the ni plugin or ini plugin with meta option. Correct mount command would be: kdb mount rest-backend-config.ini spec/sw/elektra/restbackend/#0 ni

With these three changes kdb spec-mount /sw/elektra/restbackend/#0 worked without errors for me. If this does not solve the issue for you, please post the content of config-specification.in and the output of kdb export spec/sw/elektra/restbackend/#0 ni.

Please keep the issue open in any case: I agree that the error message is not helpful. It should be something like: metadata "mountpoint" not found or even give tips that the INI needs to be mounted with the "meta" option.

[BernhardDenner]

Ok, for your three points, that are really errors in my bug report. I've cross-checked it now with my terminal history:

You use config-specification.ini from the src dir, but you should use the one from the build dir. You might have wrongly substituted values. Correct copy command for me is: cp ~build/src/tools/rest-backend/config-specification.ini /usr/local/share/elektra/specification/rest-backend-config.ini

The substitution was just mentioned to reproduce the problem without requiring to build the rest-backend (having to install its deps). On the REST-backend target system I've used the generated one.

The path you entered is non-default: Its /usr/local/share/elektra/specification/ (without s at the end, see above).

That was a mistake in my bug report, of course it is /usr/local/share/elektra/specification/

The specification file needs to be mounted with the ni plugin or ini plugin with meta option. Correct mount command would be: kdb mount rest-backend-config.ini spec/sw/elektra/restbackend/#0 ni

Same here: I've forgotten to add the ni in my bug report. I've done the mount of the spec file with the ni plugin.

I've updated my bug report accordingly.
The problem still exists for me. I've done all my tests on newly installed systems (within a container). So no existing elektra configuration/file/mounts existed before.

I've attached my results of kdb export spec/sw/elektra/restbackend/#0 ni
export_spec_restbackend_ni.ini.gz

The used config-specification.ini.in is exactly the one found in this repo config-specification.ini.in

On Ubuntu Elektra was built using the following command

export LANG=en_US.UTF-8 && \
    mkdir /root/build && cd /root/build && \
    cmake -DBINDINGS=ALL -DPLUGINS=ALL /root/libelektra && \
    make -j4 && \
    make run_all || true && \
    make install && \
    ldconfig

On the Debian 9 system, I've used the scripts/build-homepage script.

[markus2330]

Okay, thank you, now its more clear. I'll need a Debian 9 to reproduce.

Can you bisect which commit introduces the issue? (Have you automated the creation of containers?)

As workaround you can simply not use kdb spec-mount, in the end kdb spec-mount also does nothing but a kdb mount (with the collected plugins as arguments). But it seems to be a serious bug, we should fix it for the release. Maybe something is even broken in the resolver? A good way forward would be to know which commit caused the problem.

I've done a short verification if this triggers the bug with the help of the Ruby bindings

And, did it?

[BernhardDenner]

Can you bisect which commit introduces the issue? (Have you automated the creation of containers?)

I was already afraid you will ask for that ;)
Yes I can do that.

And, did it?

Yes, it did.

As far as I can tell: if SpecReader gets a keySet with the contained cascading keys it returns wrong values for the backend. Once the cascading keys are removed from the keySet SepcReader works as expected.

[markus2330]

I was already afraid you will ask for that ;)
Yes I can do that.

Thank you.

The problem might be related "spec: also create default keys" 670cd01115fa460c7e6476e8ec399eb4418bd143 (or more precisely 427a40a5e0ba710924d45eac995cfa89e6c682f2). They look like cascading keys.

But from the error I was more thinking in the direction of 38e27eb28dbf70d7df7ab1f6631482cffa13063f (and other commits heavily changing the resolver).

Hopefully the commits help you to faster locate the problem.

What makes no sense for me is why it seems to be only reproducible for Debian 9. Did you globally mount spec (can you do kdb rm -r system/elektra/globalplugins/ or kdb global-mount and still reproduce)?

[BernhardDenner]

What makes no sense for me is why it seems to be only reproducible for Debian 9.

It's not a Debian9 only issue. I've reproduced this in Ubuntu 16.04 too.

Did you globally mount spec (can you do kdb rm -r system/elektra/globalplugins/ or kdb global-mount and still reproduce)?

I thought spec is mouted globally by default, isn't it?
system/elektra/globalplugins/ does not exits and kdb global-mount does not change any thing.
Doing a kdb global-mount spec results in

The command kdb global-mount terminated unsuccessfully with the info:
It is not allowed to insert the same plugin (spec#spec) again!
Try to add other plugins or other refnames (part after #) instead.
Please report the issue at https://issues.libelektra.org/

[markus2330]

It's not a Debian9 only issue. I've reproduced this in Ubuntu 16.04 too.

Yes, I only meant the hypothesis that the error could stem from the Debian versions: 8 seems to work and 9 fails. But maybe the Debian version has nothing to do with the issue and there is something else different on my system (which makes it work). It seems to not depend on the content of the KDB, I just retried after kdb stash.

I thought spec is mou[n]ted globally by default, isn't it?

Yes.

It would be great if you can incorporate your script (for the bisect) to the test cases afterwards (to avoid regressions).

[BernhardDenner]

After lots of compiling, git bisect finally ended with the following result:

427a40a5e0ba710924d45eac995cfa89e6c682f2 is the first bad commit
commit 427a40a5e0ba710924d45eac995cfa89e6c682f2
Author: Thomas Waser <[email protected]>
Date:   Sun May 21 16:10:25 2017 +0200

    spec: also create default keys

:040000 040000 2c85c3820a58fd3905bb474863be562ff6306832 ba4a27d3c8886af12b301d9a23f82318dbfe1211 M  src
bisect run success

So it seems 427a40a5e0ba710924d45eac995cfa89e6c682f2 introduced the issue, as you already guessed ;)