Fluentd-kubernetes-daemonset: Help in building a new image with fluentd.conf file change

[lp060]

Hi,

Can anyone please help me with the procedure to build a new image if I make some code changes to fluentd.conf to pass new config params? I have modified fluentd plugin for kinesis to pass new config params. Need to build a new image with the changed plugin code.

[hobbsh]

You can just use a configmap to override the config that ships with this image - no need to build your own

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-system
  labels:
    k8s-app: fluentd
    version: v1
    kubernetes.io/cluster-service: "true"
data:
  fluent.conf: |-
    YOUR RAW CONFIG

Then mount it in the same place the image expects it, which is /fluentd/etc/

        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: fluentd-config
          mountPath: /fluentd/etc/
          readOnly: false
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentd-config
        configMap:
          defaultMode: 0666
          name: fluentd-config

[lp060]

Hi,

Thanks for the response. But I have made code changes in the fluent plugin for kinesis itself. So, I want my modified fluentd plugin to be picked up and not the gem from rubygems.org

[okkez]

1. Edit Gemfile to install your customized fluent-plugin-kinesis
   
   
   
   • maybe you can install it from git repository
   
   
2. Build your own docker image

See https://bundler.io/man/gemfile.5.html for more details about Gemfile.

[osimmasgard]

You can just use a configmap to override the config that ships with this image - no need to build your own

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-system
  labels:
    k8s-app: fluentd
    version: v1
    kubernetes.io/cluster-service: "true"
data:
  fluent.conf: |-
    YOUR RAW CONFIG

Then mount it in the same place the image expects it, which is /fluentd/etc/

        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: fluentd-config
          mountPath: /fluentd/etc/
          readOnly: false
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentd-config
        configMap:
          defaultMode: 0666
          name: fluentd-config

Hi hobbsh

was this really working for you? When I do exactly as you say I get sed: couldn't open temporary file /fluentd/etc/sedIzRrjF: Read-only file system

Thanks
Olle

[shiftux]

similar question as @osimmasgard
i get sed: cannot rename /fluentd/etc/sediSCOgt: Device or resource busy

[repeatedly]

How about this? https://github.com/fluent/fluentd-kubernetes-daemonset#disable-sed-execution-on-elasticsearch-image

[lp060]

volumeMounts: - name: varlog mountPath: /var/log - name: optvarlog mountPath: /opt/var-log - name: osseclog mountPath: /var/ossec/logs - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true - name: fluentd-config mountPath: /fluentd/etc/config.d ports: - containerPort: 24224 protocol: TCP volumes: - name: varlog hostPath: path: /var/log # In order to send same source to different indexs we need symlink. # Mounting var/log again will give similar behaviour - name: optvarlog hostPath: path: /var/log - name: osseclog hostPath: path: /var/ossec/logs - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers - name: fluentd-config configMap: name: fluentd-config
It works for me with the above configuration in my daemonset yaml where fluentd-config is my fluentd configmap

[kooboo-binbin]

I use image: fluent/fluentd-kubernetes-daemonset:v1.7.3-debian-elasticsearch6-1.0
and add a new environment then it works

• name: FLUENT_ELASTICSEARCH_SED_DISABLE
  value: "true"

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue was automatically closed because of stale in 30 days