Libelektra: FreeBSD: UBSan & failing tests

[dominicjaeger]

When compiling Elektra on FreeBSD 11.3 with ENABLE_ASAN ON I get

/usr/bin/ld: cannot find -lubsan
c++: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

FreeBSD 11.3 has clang version 8.0.0. Clang 8 documentation mentions

Make sure to use clang++ (not ld) as a linker, so that your executable is linked with proper UBSan runtime libraries

According to the error message, however, we use ld.

@sanssecours you mentioned ASAN in #3067. Do you use it for Mac only or did you get it working for FreeBSD as well?

Updated list of errors

This list is updated with knowledge from the following comments. Numbers are for multiple reports.

• [ ] testlib_pluginprocess 2
• [ ] testmod_filecheck
• [ ] testmod_iconv
• [ ] testmod_process
• [ ] testmod_yamlcpp 2
• [ ] testshell_markdown_msr_s

[sanssecours]

@sanssecours you mentioned ASAN in #3067. Do you use it for Mac only or did you get it working for FreeBSD as well?

I only used it on macOS and Linux. Probably we have to use a setup similar to the one of macOS on FreeBSD. I will check, if I am able to fix the build problems.

[sanssecours]

If you want, you can rebase on the branch 😈 of my copy of the repository, which should fix the build problems. Currently at least the tests:

• testlib_pluginprocess,
• testmod_filecheck,
• testmod_iconv,
• testmod_process, and
• testmod_yamlcpp

fail on FreeBSD, if we enable ASan, though.

[dominicjaeger]

As mentioned in #3067, testlib_pluginprocess fails after 1500 seconds due to Timeout without ASAN in make run_all. I didn't get copy&paste to work in the VM yet, sorry for errors from typing it out

105/105 Test #18: testlib_pluginprocess ........... ***Timeout 1500.13 sec
Finished loading plugin dump (in elektraPluginOpen at src/libs/elektra/plugin.c:308)
parentCommandPipe[1] has file descriptor 6 (in elektraPluginProcessInit at src/libs/pluginprocess/pluginprocess.c:511)
parentCommandPipe[0] has file descriptor 5 (in elektraPluginProcessInit at src/libs/pluginprocess/pluginprocess.c:511)
parentCommandPipe[1] has file descriptor 8 (in elektraPluginProcessInit at src/libs/pluginprocess/pluginprocess.c:511)
...
parentPayloadPipe[0] has file descriptor 7 (in elektraPluginProcessInit at src/libs/pluginprocess/pluginprocess.c:512)
...
Child: Wait for commands on pipe /dev/fd/5 (in elektraPluginProcessStart at src/libs/pluginprocess/pluginprocess.c:135)
Parent: Sending data to issue command 1 it through pip /dev/fd/6 (in elektraPluginProcessSend at src/libs/pluginprocess/pluginprocess.c:292)
Add Error C01100: Could not pen file /dev/fd/6 for writing. Reason: Operation not supported (in elektraDumpSet at src/plugins/dump/dump.cpp:287).
Parent: Waiting for the result now on pipe /dev/fd/9 (in elektraPluginProcessSend at src/libs/pluginprocess/pluginprocess.c:302)

you can rebase on the branch 😈 of my copy of the repository

I still have to try that.

[dominicjaeger]

If you want, you can rebase on the branch 😈 of my copy of the repository, which should fix the build problems.

So after I managed to download your branch (see screenshot) I can confirm that the build

cmake -DBUILD_FULL=ON -DENABLE_ASAN=ON ..
make -j 6

works. I got some failing tests as well

• 18 - testlib_pluginprocess
• 108 - testmod_yamlcpp
• 152 - testshell_markdown_msr_s

[markus2330]

Thank you for working on this! Can you make a list of failing tests in the top post of this issue?

Why are these tests not failing on our build server?

[dominicjaeger]

Thank you for working on this! Can you make a list of failing tests in the top post of this issue?

Done.

Why are these tests not failing on our build server?

I have no idea. I hope I can take a look at it soon.

[sanssecours]

Why are these tests not failing on our build server?

That is the case, since we have no FreeBSD build job that uses AdressSanitizer (ENABLE_ASAN=ON).

[markus2330]

Done.

Thank you! If you use - [ ] we can tick what was done and GitHub will also know how many of these tests are already fixed.

That is the case, since we have no FreeBSD build job that uses AdressSanitizer (ENABLE_ASAN=ON).

So the easiest solution is to say ENABLE_ASAN is not supported in FreeBSD. (To be updated in docu.)

Do these test cases above also fail without ENABLE_ASAN?

[dominicjaeger]

Do these test cases above also fail without ENABLE_ASAN?

Using just the defaults, yes.

git reset --hard HEAD
HEAD is now at 368978755 YAML CPP: Ignore container overflows on FreeBSDpwd
cmake -DBUILD_FULL=ON -DENABLE_ASAN=OFF ..
make -j 7 && make run_all
...
Total Test time (real) = 1621.23 sec

The following tests FAILED:
      18 - testlib_pluginprocess (Timeout)
    154 - testshell_markdown_msr_syntax (Failed)
Errors while running CTest
*** Error code 8
...

[markus2330]

It is always good to keep separate problems in separate issues. The ASAN issue seems to have nothing to do with the testlib_pluginprocess timeout. Maybe there is some incompatibility of how FreeBSD handles pipes.

@sanssecours Any thought why testshell_markdown_msr_syntax is broken?

[sanssecours]

@sanssecours Any thought why testshell_markdown_msr_syntax is broken?

Looks like FreeBSD’s version of grep does not support the regex syntax \s, which we currently use in markdown_shell_recorder.sh. PR #3097 should fix the problem.