Cloud-on-k8s: Custom elasticsearch configuration option should be available

[Sakib37]

Proposal

currently Elasticsearch loads the configuration from /usr/share/elasticsearch/config/elasticsearch.yml file inside the pod. It is not convenient to pass a custom elasticsearch.yml file as some configurations are not static in that file. For example-

cluster:
  initial_master_nodes:
  - pod-1
  - pod-2
  - pod-3

It will be highly convenient if Elasticsearch could load its configuration from additional .yml files as well.

Use case. Why is this important?
If I want to set some custom configuration for monitoring in the elasticsearch, I could have passed a directory where all .yml files containing some kind of configuration for elasticsearch and Elasticsearch will load all my custom configuration from /usr/share/elasticsearch/config/custom/*.yml in addition to /usr/share/elasticsearch/config/elasticsearch.yml . Now if I put a file named monitoring.yml with the following content in /usr/share/elasticsearch/config/custom/monitoring.yml

xpack.monitoring.collection.enabled: true

Monitoring will be automatically enabled when Elasticsearch starts.

[sebgl]

Hi @Sakib37,

A common approach would be to specify any config option in the Elasticsearch specification:

apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
kind: Elasticsearch
metadata:
  name: elasticsearch-sample
spec:
  version: 7.2.0
  nodes:
  - nodeCount: 1
    name: default
    config:
      xpack.monitoring.collection.enabled: true

In case you need to make this variable for multiple clusters with different configuration, you could use one of the YAML tool available (eg. Kustomize, Helm, Jsonnet, etc.). Is it not something you can easily do?

[Sakib37]

@sebelga That was clean :+1:

[sebelga]

Pinging the right person 😊 cc @sebgl