Caddy: Caddyfile Parser Crashes by Fuzzing

[Mohammed90]

I thought it might be worth it to fuzz the parser of Caddyfile to find crasher. There were 31 crashers. Linked are the crashers texts and their outputs. The majority of them seem to be due to glob expansion.

The fuzzer is not getting past the import directive. I'll keep running it every now and then hoping it finds other stuff. If it doesn't, I'll have to wait until it's fixed. Also, let's brainstorm how to include the fuzzer. Should it be in its own repo along with its corpus? Should the corpus be in its own repo but the fuzzing code is here, and the corpus is included in the main repo as a submodule? Or not include fuzzing at all?

Warning: Lots of text....

Link to gist containing description of crashers: https://gist.github.com/Mohammed90/ea28afa044cff5e9aacca0ad0d8ece3d

[mholt]

Oh thanks! I had been wanting to get around to this, I'm glad you did!

I wonder if we can fix most of the hanging ones by restricting import to only expanding 1 glob (and erroring if there's more than 1). I can't imagine off the top of my head why more than one would be needed. Some of those fuzz inputs basically expand to the whole file system!

A few of the "slice bounds out of range" panics should be simple enough to fix.

Will you be able to build from a branch? I can submit a PR with some fixes for these so far and you can go from that.

[mholt]

@Mohammed90 I've opened PR #2097 -- could you please try fuzzing on that, and see how far it gets? :smile:

I have been thinking about a grand suite of tests for Caddy, and they would probably be in a different repo, since as you mentioned the test corpus could be quite large in some cases. Let's start with a separate repo; it's easier to merge in than it is to splice out.

[Mohammed90]

I had to create a new directory for the branch and start with a fresh corpus because it kept stumbling on the old multiple asterisks issue (fuzzing noob and still learning how to best setup go-fuzz). So if I had to create a new directory for this fix, are we supposed to clear the corpus everytime? Should I re-run go-fuzz-build after the fix?

Now it's crashing on the multiple ? characters. It seems like only disallowing multiple asterisks is a premature approach given that globbing is more than asterisk expansion. The other unique crasher in this patch is a Caddyfile like this {%}.

https://gist.github.com/Mohammed90/72c0d0635f936f43d5c9ec3df07b15ed

[mholt]

@Mohammed90 Excellent -- got those fixed in 4668f88e041048e7a8fed97ed309f2bd5bb0421f, I think. Please try the latest.

Sorry, I haven't use go-fuzz in a looong time (a couple years 😶) I don't really know how to help with it, I'm afraid...

[Mohammed90]

Here's the third patch:
https://gist.github.com/Mohammed90/1fbc4784f5a78eabd6887d24582cd0ff. This one ran for 6 and half hours and only found 8 crashers.

It's still stuck on the import directive. I checked the corpus and there are a few ones with a bunch of parentheses and braces opened and closed, so there's that. I'm thinking I should try biasing it against stuff with import, what do you think?

[mholt]

Hmm, okay, so this is getting a little ridiculous. Now it's just hanging because the glob in /* is expanding to the whole file system still -- but nobody in their right mind would import config like this. And honestly, the Caddyfile input is supposed to be trusted anyway.

So, yeah, let's skip the import directive for now and let it keep going on other things, see if it finds any other panics or interesting bugs. Thanks!

[mholt]

If the fuzzer doesn't catch anything in another day or so I'll figure most of the major parsing bugs have been worked out and probably close this by merging the PR. :+1:

[Mohammed90]

Sorry for being unresponsive. I got caught with something else. I ran the fuzzer for a few hours and no interesting crashers were found. It spent majority of the time going around my attempt at biasing it against import (it kept trying mport and such 😩). So, yeah, nothing interesting for now. I'll try playing with the fuzzer and the parser more and let you know if I find anything.

[mholt]

No problem! That's awesome though, thanks for your time! Feel free to report any other interesting crashers. :)