Fluent-bit: timestamp missing with kinesis output

[UladzimirSemiankou]

Bug Report

Describe the bug
I am running fluent-bit 1.2.2 with systemd input and AWS kinesis output. Systemd does not generate the timestamp field inside the log, but put it prior to it like this:

[0] host.unknown: [1565706755.786588000, {"TRANSPORT"=>"syslog", "PRIORITY"=>"6", "SYSLOG_FACILITY"=>"10", "SYSLOG_IDENTIFIER"=>"sshd", "UID"=>"1000", "GID"=>"1000", "BOOT_ID"=>"5601a829777c459e845852afc9bcbeaf", "MACHINE_ID"=>"e14407097f0e4283a492f816e4bf6e79", "HOSTNAME"=>"ip-172-16-5-69.us-east-2.compute.internal", "SYSLOG_PID"=>"11971", "PID"=>"11971", "MESSAGE"=>"Disconnected from 172.16.12.137 port 51494", "SOURCE_REALTIME_TIMESTAMP"=>"1565706755768498"}]

When sending such a message to es output, it works fine since the output adds the timestamp, but Kinesis output seems to strip anything outside of json, so log messages look like this:

{"BOOT_ID":"53fb93cfe3664a0180672a247a4826cd","CAP_EFFECTIVE":"3fffffffff","CMDLINE":"/sbin/auditd","COMM":"auditd","EXE":"/usr/sbin/auditd","GID":"0","HOSTNAME":"ip-172-16-14-74.us-east-2.compute.internal","MACHINE_ID":"e14407097f0e4283a492f816e4bf6e79","MESSAGE":"Audit daemon rotating log files with keep option","PID":"15321","PRIORITY":"5","SOURCE_REALTIME_TIMESTAMP":"1565700926388220","SYSLOG_FACILITY":"3","SYSLOG_IDENTIFIER":"auditd","SYSLOG_PID":"15321","SYSTEMD_CGROUP":"/system.slice/auditd.service","SYSTEMD_SLICE":"system.slice","SYSTEMD_UNIT":"auditd.service","TRANSPORT":"syslog","UID":"0"}

As a result, i dont have a date field in AWS Elasticsearch. Parsing log messages for timestamp is not an option, since they dont have a consistent place to extract it from.
Can anything be done about this?

Version used: 1.2.2 with Kinesis output plugin
Configuration:
` fluent-bit-service.conf: |-
[SERVICE]
Flush 5
Daemon Off
Log_Level info
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_PORT 2020
Parsers_File parsers.conf

fluent-bit-input.conf: |-
[INPUT]
Name systemd
Tag host.*
Read_From_Tail true
Strip_Underscores true

fluent-bit-filter.conf: |-
[FILTER]
Name modify
Match host.*
Add SYSTEMD_UNIT unspecified

fluent-bit-output.conf: |-
[OUTPUT]
Name firehose
Match *
region us-west-2
delivery_stream my-stream

Running in EKS.

[edsiper]

@mhausenblas can you take a look at this ?

[PettitWesley]

@UladzimirSemiankou @edsiper This is a duplicate of the issue posted in the AWS Kinesis Firehose Fluent Bit repo: https://github.com/aws/amazon-kinesis-firehose-for-fluent-bit/issues/9

Can we close this issue since its related to the plugin and not fluent bit?

[edsiper]

thanks @PettitWesley , closing this as a dup.