Fluent-bit: Json parsing of dockerized logs is not working properly

[randude]

Hey,
I have an app that writes json to the stdout. Docker saves the logs like this:
{"log":"{\"time\":\"14:09:26\",\"rel_time\":7197,\"loglevel\":\"INFO\",\"context\":\"http\",\"module\":\"io.ssl_ctx\",\"event_tag\":\"\",\"message\":\"loading cert cache dir\",\"dir\":\"\u002Fopt\u002Fvaultive\u002Fvar\u002Fssl\"}\n","stream":"stdout","time":"2018-06-05T14:09:26.572091825Z"}

I use the json parser on this input. I then tried to apply the parser filter to parse as json the log field but It wont work since the data isnt proper json (docker changed the encoding to a json inside json). How can I make this work and get my json fields again? Do you have some sort of pre parser json encoding filter?
Thanks.

I'm using the default docker parser from the examples.

[randude]

This how mongodb logs look like:

{"log":"2018-06-05T07:35:08.730+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.\n","stream":"stdout","time":"2018-06-05T07:35:08.73067826Z"}

This means the default parser you supplied wont work properly.

[randude]

I tried this parser:
[PARSER]
Name vtv-json
Format json
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
Time_Keep On
# Command | Decoder | Field | Optional Action
# =============|==================|=================
Decode_Field_As escaped_utf8 log
Decode_Field_As json log

But it not working :(
What am I doing wrong?

[randude]

Also tried to add a parser filter but it's not working.

[FILTER]
    Name parser
    Match vtv-proxy.*
    Key_Name log
    Unescape_Key True
    Parser json

[kyleroot]

I had the same problem. I'm now able to use the following parser, but I needed to use 0.13.2. The escaped_utf8 decoder wasn't in 0.13.0.

    [PARSER]
        Name        docker
        Format      json
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L
        Time_Keep   On
        Decode_Field_As escaped_utf8 log

[randude]

@kyleroot I'm using 0.13.2 but my problem isn't the utf8, it's the fact that i have an unscaped json inside the log variable and I cant get this to work. I had to send it via http to logstash and extract the json from there.

[kyleroot]

You're right, there are some logs I have that json within the already json docker logs. The inner json is not being unescaped correctly. This is still a problem in 0.13.4.

[SkeLLLa]

As far as I understood issue is still present in 0.14.7?

[edsiper]

Please check the following comment on #1278 :

https://github.com/fluent/fluent-bit/issues/1278#issuecomment-499583503

[edsiper]

Issue already fixed, ref: #1278 (comment)