Libelektra: Failing tests with kdb run_all in Docker container

[alexsaber]

Steps to Reproduce the Problem

When running the tests inside a Docker container like described in run_all_tests_with_docker.md

kdb run_all

2 tests are failing.

Expected Result

The tests should not fail

Actual Result

Running testmod_lua

LUA         TESTS
==================

Testing simple variable passing...
There are 1 warnings
buffer is: warnings/#00
number: 11
description: open of plugin returned unsuccessfully (reason contains plugin, see other warnings for details)
module: 
file: /home/jenkins/workspace/src/libs/elektra/plugin.c
line: 296
reason: lua
reason: 
reason: 
/home/jenkins/workspace/src/plugins/lua/testmod_lua.c:24: error in test_variable_passing: warnings in kdbOpen for plugin lua
number: 131
description: : lua error
module: : lua
at: /home/jenkins/workspace/src/plugins/lua/lua.cpp:172
reason: : module 'kdb' not found:
        no field package.preload['kdb']
        no file '/usr/local/share/lua/5.3/kdb.lua'
        no file '/usr/local/share/lua/5.3/kdb/init.lua'
        no file '/usr/local/lib/lua/5.3/kdb.lua'
        no file '/usr/local/lib/lua/5.3/kdb/init.lua'
        no file '/usr/share/lua/5.3/kdb.lua'
        no file '/usr/share/lua/5.3/kdb/init.lua'
        no file './kdb.lua'
        no file './kdb/init.lua'
        no file '/usr/local/lib/lua/5.3/kdb.so'
        no file '/usr/lib/x86_64-linux-gnu/lua/5.3/kdb.so'
        no file '/usr/lib/lua/5.3/kdb.so'
        no file '/usr/local/lib/lua/5.3/loadall.so'
        no file './kdb.so'
mountpoint: : 
configfile: : 
/home/jenkins/workspace/src/plugins/lua/testmod_lua.c:24: error in test_variable_passing: error in kdbOpen for plugin lua
/home/jenkins/workspace/src/plugins/lua/testmod_lua.c:24: fatal in test_variable_passing: could not open lua plugin
error: testmod_lua

Running testkdb_ensure

Running main() from /opt/gtest/googletest/src/gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from Ensure
[ RUN      ] Ensure.GlobalUnmount
[       OK ] Ensure.GlobalUnmount (62 ms)
[ RUN      ] Ensure.Unmount
/home/jenkins/workspace/tests/kdb/testkdb_ensure.cpp:102: Failure
Expected equality of these values:
  root.getMeta<std::string> ("warnings/#00/number")
    Which is: "79"
  "3"
error plugin didn't run
[  FAILED  ] Ensure.Unmount (76 ms)
[ RUN      ] Ensure.GlobalMount
tracer: open(0x5575e48c6b30, spec/tests/kdb/ensure = ): 0
tracer: close(0x5575e48c6b30,  = )
[       OK ] Ensure.GlobalMount (64 ms)
[----------] 3 tests from Ensure (203 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (203 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Ensure.Unmount

 1 FAILED TEST
error: testkdb_ensure

[markus2330]

@kodebach do you have any idea why testkdb_ensure fails?

[kodebach]

The Google Test failure message is pretty explicit. The error plugin didn't run, or in this case it might have run, but there were other unexpected warnings. That's why "warnings/#00/number" is 79 and not 3.

Warning 79 is "postcondition of backend was violated", no idea where that comes from. Probably the same cause as the other test failure.

EDIT: in #2782 the same error occurs and the same warning causes a few other tests to fail, so I doubt it has anything to do with testkdb_ensure in particular.

[kodebach]

Just looked at doc/tutorials/run_all_tests_with_docker.md. It says to pull the images from our repository. So my guess is the problem here is that those images use a uid/gid suitable for our Jenkins setup. If you pull the image from the repository you should only run them with --user $(id -u):$(id -g).

[alexsaber]

@kodebach thank you!
If I use the current user with --user $(id -u):$(id -g) I have another permission rights problems.
could not create configuration directory: Could not create directory "/.config", because: "Permission denied" uid: 1000, euid: 1000, gid: 1000, egid: 1000

[kodebach]

You probably have to set the CMake variables for the various KDB directories to be inside the mapped directory of the docker container. Possibly there are other configurations that work too, not sure what your exact setup is.

[alexsaber]

You probably have to set the CMake variables for the various KDB directories to be inside the mapped directory of the docker container. Possibly there are other configurations that work too, not sure what your exact setup is.

Yes, this is the first thing I tried.

cmake /home/jenkins/workspace -DBINDINGS="ALL;-DEPRECATED; -haskell" -DPLUGINS="ALL;-DEPRECATED" -DTOOLS="ALL" -DENABLE_DEBUG=ON -DKDB_DB_HOME="/home/jenkins/workspace" -DKDB_DB_SYSTEM="/home/jenkins/workspace/elektra-build-docker/.config/kdb/system" -DKDB_DB_SPEC="/home/jenkins/workspace/elektra-build-docker/.config/kdb/system" -DINSTALL_SYSTEM_FILES="OFF" -DCMAKE_INSTALL_PREFIX="/home/jenkins/workspace/elektra-install"

and

cmake /home/jenkins/workspace -DBINDINGS="ALL;-DEPRECATED; -haskell" -DPLUGINS="ALL;-DEPRECATED" -DTOOLS="ALL" -DENABLE_DEBUG=ON -DKDB_DB_HOME="/home/jenkins/workspace" -DKDB_DB_SYSTEM="/home/jenkins/workspace/.config/kdb/system" -DKDB_DB_SPEC="/home/jenkins/workspace/.config/kdb/system" -DINSTALL_SYSTEM_FILES="OFF" -DCMAKE_INSTALL_PREFIX="/home/jenkins/workspace/elektra-install"

Nothing worked. Any other ideas?

[markus2330]

Seems like the problem can only be fixed if you build the docker container correctly. See scripts/docker/README.md (docker build)

@ingwinlu do you have another idea to avoid a local docker build?

[alexsaber]

I've just tested it with my own-built docker image from scripts/docker/debian/sid/Dockerfile and --build-arg JENKINS_USERID=$(id -u) --build-arg JENKINS_GROUPID=$(id -g).

These tests failed.

testmod_lua

Any more ideas?

[markus2330]

So some progress! Can you update your tutorial?

Does testkdb_ensure produce the same output as already supported?

testmod_lua might be a known problem (for some lua versions).

[alexsaber]

@markus2330 Unfortunately, there is no progress. If you look at the first description of the problem, exactly these 2 tests were failing. Then I tried, like suggested by kodebach, to spin-up a container of your public image with my user id, and then even more tests failed. So basically, we came to the same point.

I haven't tried to spin-up a container from my own-built image with my user id, I this should not make any difference.

Does testkdb_ensure produce the same output as already supported?

Sorry, I don't understand the question. Can you please explain it in more detail?

[markus2330]

I haven't tried to spin-up a container from my own-built image with my user id, I this should not make any difference.

This is how the build server does it and there all test cases pass without failures.

Sorry, I don't understand the question. Can you please explain it in more detail?

I think you already answered it: the same test cases fail for the same reasons?

[alexsaber]

This is how the build server does it and there all test cases pass without failures.

I've just tested it like described. I tried to spin-up a container with my user id of my own-built image which was built with my user id as well. I see exactly the same 2 tests failing.

[kodebach]

Did you already try to use a clean copy of the Elektra repository (i.e. new folder and git clone again)?

Also did you build inside the container or did you only run the tests with a build from your host system?

[alexsaber]

Did you already try to use a clean copy of the Elektra repository (i.e. new folder and git clone again)?

No, I can try it.

Also did you build inside the container or did you only run the tests with a build from your host system?

I built everything inside the container. Otherwise why to bother to have Docker in the first place?

[alexsaber]

Did you already try to use a clean copy of the Elektra repository (i.e. new folder and git clone again)?
No, I can try it.

Just tried it. Exactly the same 2 failing tests. @kodebach does it work for you, if you run the tests in a container created from a self-built image? I know that the build server does the same thing. But it does not change the fact that the tests are failing anyway.

[markus2330]

@kodebach Can you please check if the ensure test case works for you in a docker container? (And how did you build this docker container?)

[alexsaber]

Thank you all for your support! The problem has been solved. The solution is described in the tutorial. I close the issue.