Fluent-bit: Support Journald (in_journald)

[edsiper]

We have get many requests to support native Systemd logging facilities (Journald). This issue aims to track the progress of such implementation.

To be implemented for Fluent Bit v0.12

[DonMartin76]

As many default docker host deployments (e.g. ACS on Azure) use this as a default log driver, this would be absolutely and completely awesome to have.

[edsiper]

Everyone,

I've pushed a new Systemd input plugin in our master branch (v0.12) which will be available on our next major release. The plugin have the following features:

• Read log messages from Journald
• Allows to specify filters in the configuration (e.g: _SYSTEMD_UNIT=docker.service)
• Limit the rate of logs processed per Systemd notification
• Tag: when used with a wildcard, it expand using the Systemd Unit name (tag host.* => host.docker.service)
• Database file: allow to keep a persistent state of the Journald cursor

Here is the documentation draft of the new plugin:

http://fluentbit.io/documentation/0.12/input/systemd.html

would you please give it a try and send some feedback ?

[git001]

I have run it on centos latest

cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)

bin/fluent-bit -i systemd -p systemd_filter=_SYSTEMD_UNIT=sshd.service -o stdout

and I got some data.

[0] systemd.0: [1497137111.000000992, {"PRIORITY"=>"6", "_UID"=>"0", "_GID"=>"0", "_SYSTEMD_SLICE"=>"system.slice", "_BOOT_ID"=>"af22dec23f624188933e70c2664da8d3", "_CAP_EFFECTIVE"=>"1fffffffff", "_MACHINE_ID"=>"56323da06e9b491caa907adfaf01401b", "_TRANSPORT"=>"syslog", "SYSLOG_FACILITY"=>"10", "SYSLOG_IDENTIFIER"=>"sshd", "_COMM"=>"sshd", "_EXE"=>"/usr/sbin/sshd", "_SYSTEMD_CGROUP"=>"/system.slice/sshd.service", "_SYSTEMD_UNIT"=>"sshd.service", "_HOSTNAME"=>"centos-512mb-fra1-01", "_CMDLINE"=>"sshd: root [priv]   ", "SYSLOG_PID"=>"7853", "MESSAGE"=>"Accepted publickey...", "_PID"=>"7853", "_SOURCE_REALTIME_TIMESTAMP"=>"1497137111988762"}]

You can find the build process in this dockerfile
https://github.com/git001/fluent-bit-test/blob/master/Dockerfile

[edsiper]

@git001 thanks for the confirmation.

Fixed.