Libelektra: try out AFL

[manuelm]

http://lcamtuf.coredump.cx/afl/

[markus2330]

Thats exactly the tool I searched for. Thank you!

With

afl-fuzz -i testcase_dir -o findings_dir bin/kdb import user/tests

(and some *.dump files in testcase_dir) I already found a crash bug (sizes larger than file are not checked)

More interesting would be to have fuzz tests that allow all API calls (kdb shell) and mounting (mounting by spec is currently only proof-of-concept), but I am afraid those kinds of tests are more involved, because then it might be necessary to specify that only method arguments and not names should be fuzzed. Would be interesting how they test php.

[manuelm]

Haven't looked into it at all but that was the tool I was referring too a couple of weeks ago

[markus2330]

Hello,

manuel:

Haven't looked into it at all but that was the tool I was referring too a
couple of weeks ago

Yes, I know. Thanks again.

Over night the tool did 5 more cycles, but the crashes it found were
variations of the first one (but maybe in other code pathes).

best regards

[markus2330]

I fixed the problem which I found in dump b817ddf9340a22b5c2ef25841d6058cec1c6e1a3 (not checking for too large allocations)

could not find any issues in testing "kdb shell" with AFL.

[manuelm]

Another fuzzer just got released: http://blog.llvm.org/2015/04/fuzz-all-clangs.html

[markus2330]

We successfully tried the fuzzers, I added the task to integrate it on the build server in #160

[markus2330]

@tom-wa AFL is the fuzzer I told you about.

So if you create a bunch of INI files in the testcase_dir, and run:

afl-fuzz -i testcase_dir -o findings_dir bin/kdb import user/tests ini

it might report crashes to be fixed.

[markus2330]

Another fuzzer:

https://sourceforge.net/projects/sharefuzz/

[markus2330]

We should check the dini/ini plugin with AFL before making it the default plugin.

[markus2330]

Seems like the INI plugins produces many crashes at the moment. I collected 202 unique crashes in https://github.com/ElektraInitiative/blobs/tree/master/configs/ini (I'll add more later.)

[markus2330]

Seems like we fixed all crashes with the INI plugin. AFL did a successful pass without any crash or hang.

[markus2330]

@sanssecours Maybe you can also try out AFL for the new storage plugin #2330?

[sanssecours]

Maybe you can also try out AFL for the new storage plugin #2330?

I already wrote that I will check the YAML storage plugins with AFL in the current version of my thesis. That said, it really depends on the other parts of the comparison, if I will have time to test the plugins with AFL.

[markus2330]

Maybe checking the final YAML storage is enough. Doing these checks for all the plugins is low priority.

We should add the info if we used a fuzzer in the contract, see #666