Fluent-bit: Support for filtering on kubernetes labels?

[dpgeekzero]

I may be missing something, but I only want to export logs for some subset of my kubernetes pods.

Can I use the filtering to only pass through logs from pods with a label like app=iwanttolog?

[michiel]

The grep filter might be what you're looking for. Something like this perhaps?

[FILTER]
    Name   grep
    Match  k8s.*
    Regex  app iwanttolog

https://fluentbit.io/documentation/0.13/filter/grep.html

[jinchaoxiang]

这样写不行

[jinchaoxiang]

can't work

[michiel]

If you need flexibility in matching the keys or values, you might also consider the following solution. In 0.14 (not yet released), you can apply conditions with regex matchers to the modify filter. This PR has the documentation.

[FILTER]
    Name   modify
    Match  k8s.*
    Condition Key_value_matches log regex_to_match.* 
    Set DROP_THIS true

[FILTER]
    Name   grep
    Match  k8s.*
    Regex  DROP_THIS true

[rlguarino]

@michiel how can we try 0.14? Is there a docker image build somewhere. Is there an ETA?

[michiel]

@rlguarino This build is from the current master at 6764ac7 , it should have everything you need.

• Tag : travis-122
• Docker : https://hub.docker.com/r/sddmelb/fluent-bit/tags/
• Build history : https://travis-ci.org/michiel/fluent-bit/builds/413818971

[MohdAhmad]

@michiel thanks for sharing this. Quick question, any thoughts on what the more efficient method is? The aim for me is drop all records outside of the kube-system namespace.

[kaspernissen]

I'm looking for a similar thing. I'm trying to discard logs that doesn't match a label on pods. I have tried many different combinations of conditions without any luck. I only want to pass logs a long that has the following label k8s.labels.type equals to service.

https://github.com/michiel/fluent-bit-docs/blob/172d0e8538603229e61b7e1b9cc52b7933569a72/filter/modify.md

[FILTER]
  Name                kubernetes
  Match               *
  Kube_URL            https://kubernetes.default.svc:443

[FILTER]
  Name     modify
  Match    *
  Rename   kubernetes k8s

[FILTER]
  Name     modify
  Match    *
  Condition Key_Value_Equals k8s.labels.type service
  Set testing MATCH

If I try the condition Key_exists k8s then i get the field testing with the value MATCH. But how to test nested conditions?

I'm migrating from fluentd, where we could do something like this with the record_modifier:

<filter application.**>
  @type record_modifier
  <record>
    disable "${(record['kubernetes']['labels']['type'] == 'service') ? false : true}"
  </record>
</filter>

Is there any way to do something similar with fluent-bit?

[thspinto]

I found a workaround fluent-bit's root key limitation for this case. This might help you @kaspernissen.

My idea is to copy the fields I want to operate on to a new key. Then I lift all the relevant fields on the kubernetes_copy_ prefix. After I filter and modify the record I drop all fields with the kubernetes_copy prefix. This way I get to do all my pipeline operations directly on fluent-bit and get a clean result at the end.

[FILTER]
  Name                   kubernetes
  Match                  *
  Kube_URL               https://kubernetes.default.svc:443

[FILTER]
  Name                   modify
  Match                  *
  Copy                   kubernetes kubernetes_copy

[FILTER]
  Name                   nest
  Match                  *
  Operation              lift
  Nested_under           kubernetes_copy
  Prefix_with            kubernetes_copy_

[FILTER]
   Name                  nest
   Match                 *
   Operation             lift
   Nested_under          kubernetes_copy_labels
   Prefix_with           kubernetes_copy_labels_

[FILTER]
   Name                  grep
   Match                 *
   Exclude               kubernetes_copy_container_name fluent

[FILTER]
   Name                  modify
   Match                 *
   Rename                kubernetes_copy_container_name      app
   Rename                kubernetes_copy_namespace_name  env
   Remove_wildcard       kubernetes_copy