Libelektra: document required environment for running tests

[markus2330]

Plugins have several assumptions towards the environment so that their tests can run. (Assumptions that cannot be checked while compiling.) Most of the plugins check for these assumptions and disable tests so it is important that it is documented what is needed to fully run all tests. Please help me for gathering the full list. We have:

• the UID running the tests must have a home directory (see #1881)
• mounted /dev (to have stdin and stdout for import & export test cases)
• running dbus daemon (dbus plugins, @waht should know more about that)
• gpg2 binary (@petermax2 should know more about that)
• semlock (tests currently disabled because the detection of the assumptions did not work reliable, @KurtMi should know more about that)
• clang-reformat-5 to 7 to check formatting
• installed Elektra (for gen + external test cases)
• checkbashisms for tests/shell/check_bashisms.sh
• no keys below /tests (actually at the moment it is also /test /examples, .. among others, see #1887)
• and of course the big topic that many tests need specific tools/plugins in order to work, see #944
• pkg-config (check_external.sh and check_gen.sh)
• git (tests/shell/check_formatting.sh)
• gcc (check_gen.sh)

Should we document this in doc/TESTING.md (one central place) or individually in the contract of the plugins? (And write a script that collects all information together.)

[petermax2]

gpg2 binary (@petermax2 should know more about that)

Either version 1 or version 2 is required (gpg or gpg2 must be executable by the user, that runs the tests). This is the only runtime requirement for the crypto and fcrypt plugins.

Personally I prefer to store the info at doc/TESTING.md. The information is not being changed that often. Also one has a central place to look for when setting up an environment for running the tests.

[waht]

Yes, for the dbus plugin tests a D-Bus daemon needs to be running. At least one bus (either "system" or "session") needs to be available.

I agree with @petermax2: doc/TESTING.md seems like the better option.

[markus2330]

@waht Thank you for the quick answer! Does it test different things if system and/or user sessions are started?

[waht]

You're welcome :)
No, the tests are the same since on MacOS only a "session" bus is available (from homebrew) and on Linux the "system" bus is usable on CLI (e.g. build server or container) - it seems like the "session" bus is entangled with X11.

In a (docker) container with the "session" bus started the connection fails:

root@c41038f06544:/# dbus-daemon --session --fork                                                                      
root@c41038f06544:/# dbus-monitor --session
Failed to open connection to session bus: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11   

[markus2330]

See also #1820 for different approaches (avoiding manual docu) and #1899 for a very similar to discussion: what to do if we know a test will fail but without a way to detect it from within the program but only within CMake.

[markus2330]

Thank you for your help!

If additional envs are needed, please document them in TESTING.md.