Fluent-bit: How to use few fields in grep filter?

[bat9r]

I`m using fluent-bit:0.13.0, i want grep "warns", "errors" and etc. from multiple fields using grep filter. How i can do it?
This is part of my config.
filter-kubernetes.conf: | [FILTER] Name kubernetes Match kube.* Kube_URL https://kubernetes.default.svc.cluster.local:443 Merge_JSON_Log On tls.verify Off tls.debug 4 [FILTER] Name grep Match kube.* Regex log|msg (?i)warn|error|exception|critical|warning|err
How to grep from "log" and "msg" field if they comply regex?

[edsiper]

@bat9r It depends on the format of your incoming logs, I would put the grep filter before kubernetes filter, so you let pass the only ones you care about.

[bat9r]

@edsiper
Thank you for answer.
Soo, i can't "care about" for two fields? Or i could do it from other side (an application which aggregate logs)?

[michiel]

A workaround might be (on 0.14 - currently unreleased) to apply the modify filter twice with conditions. This allows you to create an OR condition where a record is tagged with k/v _keep:true_ if it matches either condition. You can then apply the grep filter to keep or drop the record based on that k/v.

   [FILTER]
       Name           modify
       Match          kube.*
       Condition      Key_value_matches log REGEX_TO_MATCH
       Set            keep true

   [FILTER]
       Name           modify
       Match          kube.*
       Condition      Key_value_matches msg REGEX_TO_MATCH
       Set            keep true

   [FILTER]
       Name           grep
       Match          kube.*
       Regex          keep true

[bat9r]

@michiel Thank you, great idea :)

[viveksinghggits]

I`m using fluent-bit:0.13.0, i want grep "warns", "errors" and etc. from multiple fields using grep filter. How i can do it?
This is part of my config.

filter-kubernetes.conf: |
   [FILTER]
       Name           kubernetes
       Match          kube.*
       Kube_URL       https://kubernetes.default.svc.cluster.local:443
       Merge_JSON_Log On
       tls.verify     Off
       tls.debug      4
   [FILTER]
       Name           grep
       Match          kube.*
       Regex          log|msg (?i)warn|error|exception|critical|warning|err

How to grep from "log" and "msg" field if they comply regex?

Hi @bat9r ,
does line Regex log|msg (?i)warn|error|exception|critical|warning|err mean that I want to filter the message where either log or msg field is among warn|error|exception|critical|warning|err.
If yes, I wanted to filter the logs where container_name is either nifi or redis I used below config, which doesnt seem to work
Regex CONTAINER_NAME nifi | redis
any suggestion ?