Helm-charts: Help in how to override filebeatConfig

[eyesmoker]

Hello,

I am using helm --set to override filebeatConfig at run time https://github.com/elastic/helm-charts/blob/master/filebeat/values.yaml

Please advise.

Error: template: filebeat/templates/daemonset.yaml:29:27: executing "filebeat/templates/daemonset.yaml" at <include (print .Template.BasePath "/configmap.yaml") .>: error calling include: template: filebeat/templates/configmap.yaml:13:35: executing "filebeat/templates/configmap.yaml" at <.Values.filebeatConfig>: range can't iterate over

Value trying to override:

                      filebeat.yml: |
                        filebeat.inputs:
                        - type: docker
                          containers.ids:
                          - '*'
                          processors:
                          - add_kubernetes_metadata:
                              in_cluster: true

[jmlrt]

Hi @eyesmoker,
We don't have enough informations in the issue description.
Also we prefer having questions like that submitted in Elastic forums: https://discuss.elastic.co/

Can you create a topic on the forum?
Don't forget to paste the full helm command you are typing so we can help you.

[awegmann]

Even the issue is old, I want to annotate this.

We encountered the same problem. We wanted to overwrite the filebeat.yml with a modified one. What we tried was, to create a yaml file with the content of the variable filebeatConfig and set this with --set-file on the install command. The result was exactly your error message.

Our solution:

Create a config yaml file like the values.yaml of this repo and remove all default values you don't want to change.

Our file look like this:

filebeatConfig:  
  filebeat.yml: |
    - type: container
      paths:
        - /var/log/containers/*.log
      json.keys_under_root: true
      json.add_error_key: true
...

Use this file and -f with the helm install command and you can overwrite the default value with this.

My explanation (btw: I'm not a helm expert):

From the documentation of helm:

-f, --values strings               specify values in a YAML file or a URL (can specify multiple)

It looks like that the handling of values is different in helm if you specify variables with -f and --set / --set-file.
I guess with -f the yaml file is parsed and even the values of filebeatConfig are an iterable collection in this case. With --set / --set-file the value is just a simple string, which is not an iterable key/value which is assumed in the template.

If such a yaml file should be used it needs to be parsed before with fromYaml but I guess this would interfere with the behavior of values set with -f.

See also: https://stackoverflow.com/questions/57658574/helm-cant-iterate-over-range-from-set-file

@jmlrt It would be good if you can verify my solution and maybe add a hint in the documentation that filebeatConfig cannot be set with --set-file or --set.