Fluent-bit: Output plugin

[prashantvicky]

Hi @edsiper ,
Just wanted to know is it possible to use multiple output plugins at the same time , like we forward data to elastic search as well as syslog server.
Any example will be great.

Regards
Prashant

[zfouts]

@prashantvicky - you can, please look into out_copy -- https://docs.fluentd.org/v1.0/articles/out_copy

You can define multiple outputs via copy.

[prashant5375]

I am using fluent-bit not Fluentd , can u provide the solution for that @zfouts

[nokute78]

It's easy, just write multiple match cases.

This is an example .1 input (in_dummy) and 2 outputs (out_stdout and out_file)

[INPUT]
    Name dummy
    Tag  case.multi

[OUTPUT]
    Name stdout
    Match *.multi

[OUTPUT]
    Name file
    Match *.multi

output is like this.

[taka@localhost build]$ bin/fluent-bit -c multi.conf 
Fluent-Bit v0.14.0
Copyright (C) Treasure Data

[2018/08/24 17:33:23] [ info] [engine] started (pid=10857)
[0] case.multi: [1535099604.001631211, {"message"=>"dummy"}]
[1] case.multi: [1535099605.001801261, {"message"=>"dummy"}]
[2] case.multi: [1535099606.000245533, {"message"=>"dummy"}]
[3] case.multi: [1535099607.002233306, {"message"=>"dummy"}]
^C[engine] caught signal (SIGINT)
[2018/08/24 17:33:28] [ info] [input] pausing dummy.0


[taka@localhost build]$ cat case.multi 
case.multi: [1535099604.001631, {"message":"dummy"}]
case.multi: [1535099605.001801, {"message":"dummy"}]
case.multi: [1535099606.000246, {"message":"dummy"}]
case.multi: [1535099607.002233, {"message":"dummy"}]

[prashantvicky]

Hi @nokute78
Do we have any output plugin to flush logs to syslog server. As if i use forward plugin to send data to syslog server , it sends some data for it create multiple file like given below

915608384Z"}.log
"resourceVersion\".log
,\"pod-template-generati
883069576bef99b9d7671a49
beTime\".log
Class\".log
on-hash\".log
-bit-gcklh\",\"uid\".log
aa2e-005056ae6504\n","st
f-a934-11e8-aa2e-005056a
01294b412235c49fe49a1ff0
*o.log
nt-bit-rmgvc.log
6T13.log
1Z"}.log
2018.log
Lpy.log
ubernetes.log
id.log
43f962bc58d23b3e11f3e139
ent-bit-logging.log
u-vm5.log
-generation.log
kubernetes.io.log
eneration.log
r_name.log
status\".log
iner_name.log
tu-vm5.log
derr","time".log
untu-vm5.log
11e8-aa2e-005056ae6504.l
2168194661.log
34.log
evision-hash.log
(#031#025.log

0309.log

0ba720a6f40751b35.log

010V.log

n$.log
n>.log
+y.log
+%9.log
nt-bit-logging.log
ime".log
Vq.log
6\",\"_t\n","stream".log
;.log
)#014.log
kZ.log
[email protected]
K#010.log
KB.log
vH.log
stream".log
l7&.log
l#034#027.log
l#007.log

015K.log

015fe.log

E].log
E#033.log
e0ffa79a801294b412235c49

037#006h.log

UcT.log
Gi.log

005{"log".log

?JM.log
?d.log
0ffa79a801294b412235c49f
docker_id.log

020*.log

f7S.log
fx.log
+1.log
mA.log
36.log
DY.log
kubernetes.log
B{N.log
=..log

024O.log

Y'.log
[email protected]
Y#0052.log

000#017.log

[email protected]

020.d.log

ration.log
pod_id.log

037#005J.log

jna.log

026x.log

1ff0ba720a6f40751b35.log
XB.log
hK.log
Dk.log
,\"status\".log
n&.log
n7#027.log
n#026n.log
K#023#027.log
K.#000.log

023).log

023d-.log

023#006.log

eYc.log
eo.log
eK.log
e0.log
Guh.log
{y#014.log
{#023^.log
{V-.log
h-2018.08.26\",\"_type\"
We.log
f}.log
)`.log
To.log
T3#007.log
T#021.log
T".log
T_.log
TC.log
F!.log
Fc.log
n","stream".log
nd.log

003#005.log

007D.log

020#013^.log

020L.log

0202708505eba1fff4eb1282
neration.log
,k.log
,].log
883e0ffa79a801294b412235
rn#021.log
r2+.log
5d6883e0ffa79a801294b412
bug].log
zv.log
e\",\"_id\".log

0010.log

025o.log

025od.log

025o&#030.log

021M.log

015?#005.log

011#022.log

011#021.log

005#004.log

015>.log

036ME#025.log

025n.log

021M^.log

;e.log

036M.log

036M#034W.log

025oE#011.log

001'{"log".log

b#016.log

v.log
U#024.log
R#014.log
Rrh.log
RP.log
Ra.log
ce.log
it.log
@"Z.log

022p.log

022#014.log

022Ni.log

]s#030.log
]J.log
]e.log
K#022.log
K1.log
se\n","stream".log
SB.log
s2.log
roE.log
tes.log
Q#037.log
PM.log

024F.log

024~2.log

024mm.log

HuW.log
H;D.log
H#030.log
y#.log
)X.log
)x#002.log
@|.log
v~B.log
{#036.log
q#032.log
pV.log
ASz.log
)eh.log
)#000!.log
)B.log

026WM.log

026$d.log

026r.log

026F.log

fluent-bit-rmgvc.log
19510\n","stream".log
wO.log
w#014k.log
661.log

000".log

000F.log

94661.log
ask=0x7fc14b854620.log
e#010.log
e#.log

002{.log

007#017.log

007#017C#007.log

007#016.log

030j#016I.log

030i.log

007#017#037.log

$c+.log
$c#002.log

030j+<.log

030j>d.log

030j#034.log

024I.log

0201.log

014#034.log

007#020#000.log

007#017f3.log

$cf.log
z=.log
Can you please help me with this.

[nokute78]

Fluent-bit doesn't support out_syslog currently. (feature request also came like #595)

There is 3rd party plugin "pivotal-cf/fluent-bit-out-syslog".
https://github.com/pivotal-cf/fluent-bit-out-syslog

It may help us. (I've never used it.)

[prashantvicky]

@nokute78 , i have tried that pivotal , not working.
@edsiper , @nokute78 Do we have any timelines for this , by when can we have this feature.
Even though if you have any dev branch which supports this , will be gud

[edsiper]

syslog output requirement is filed at #595.

Since main questions have been addressed I am closing this issue.