Fluent-bit: pgsql: unterminated quoted string

[tuckerconnelly]

Bug Report

Describe the bug

When running the new pgsql plugin, an error with unterminated quoted strings prevents logs from being inserted.

To Reproduce

Log:

[2020/03/28 23:32:37] [ warn] [out_pgsql] ERROR:  unterminated quoted string at or near "'[{"timestamp":1585438356.853608,"log":"10.0.1.242 - - [28/Mar/2020:23:32:36 +0000] \"GET / HTTP/1.1\" 200 0 \"\" \"kube-probe/1.15+\"\n","stream":"stdout","time":"2020-03-28T23:32:36.853607625Z","kubernetes":{"pod_name":"harbor-harbor-registry-74d8dc4bd6-x77xk","namespace_name":"devops","pod_id":"42d92d06-0942-4569-bda8-3a396050428e","labels":{"app":"harbor","chart":"harbor","component":"registry","heritage":"Helm","pod-template-hash":"74d8dc4bd6","release":"harbor"},"annotations":{"checksum/configmap":"a01fc789d0434a52795328e885e09ab678688c5bb02a6ec05bf526213ba91ce1","checksum/secret":"a805b94ba2076b48a9156e9515261d4f0a3893e5cc563450856495f4767c56e6","checksum/secret-core":"c69af3373980da3a15e346a4d3b762fc962b17e1e0d873b22a0a3ab50677c090","checksum/secret-jobservice":"f03cebad4f1c10d3bc48a8b45d5985c13126201ca15654f2748b073ab887f62d","kubernetes.io/psp":"eks.privileged"},"host":"ip-10-0-1-242.us-west-2.compute.internal","container_name":"registry","docker_id":"bc52b318da5fe41ab87f2db3301ea8862f6762ad85b9fc0e534f58ae662bdcfe","container_hash":"goharbor/registry-photon@sha256:3ee2e7dc7f93d9b922d382b8454fd017288d8eb8fdfdd022ade81e7c9f171723","container_image":"goharbor/registry-photon:v2.7.1-patch-2819-2553-v1.10.1"}}]"

Expected behavior

Logs to be inserted.

Your Environment

• Version used: 1.4.1

Additional context

I think single quotes just need to be escaped by being doubled.

Left a comment on PR here: https://github.com/fluent/fluent-bit/pull/1931#discussion_r399721410

[sxd]

@edsiper can you assign this issue to me so I can keep the track? I'm working on a solution now

[sxd]

@tuckerconnelly Ihave a patch but still not able to reproduce this error, do you have a file that I can read and will be able to reproduce this error? or can you compile and reproduce? I'm able to build a package with the fix if you need it to test

[tuckerconnelly]

Thank you @sxd!

Pulled your branch and got it working! (Was super easy thanks to the Docker setup, props to the maintainers of fluent-bit).

The only thing was the 60 here needs to be a 61, or it truncates the last ): https://github.com/sxd/fluent-bit/commit/3996a8c43ec21c2c0aff800249a306f8cdb77cbb#diff-016e856dc50c081c3a41c6828d14d4b2L247 (which, if you don't mind a little constructive feedback to make your code as good as possible, is a bit of a magic number and could benefit from a comment explaining it).

You're probably aware already, but your commit also fixes a security vulnerability where an attacker might be able to inject SQL into the logs to get access to the database, which is great.

I pushed a docker image: tuckerconnelly/fluent-bit:sxd-pgsql-fix if anyone runs into this issue and wants to use it now.

Thanks again Jonathan!

[sxd]

@tuckerconnelly Thanks!

Awesome that it works! and yes I have to fix a few things with the patch since it's just a PoC that it will be fixed with that. I'll work on that today and send it to fix the issue ASAP.

Thanks for reporting this! will let you know when you will be able to see it on branch master.

Best Regards!

[sxd]

@tuckerconnelly hi! as you can see there's a PR already there =) if you want to take a look.

[edsiper]

Fixed with #2062, the fix will be part of the next v1.4.2 release.