Cloud-on-k8s: Document a simple Job to initialize Elasticsearch with desired API calls

[sebgl]

As a stop-gap before more features can (maybe eventually) support declarative file-based configuration (ILM, SLM, etc.), we could document a very simple Kubernetes Job that:

1. waits until Elasticsearch is available
2. performs a bunch of API calls
3. exit 0 (and never runs again)

[mkretz]

I recently hit the issue of the missing support for declarative configuration. Here's the Kubernetes Job I came up with. It waits for Elasticsearch to become ready and then configures an ILM policy and an index pattern which links to the policy. Feel free to use it as an example or let me know if I should directly insert it somewhere in the docs.

apiVersion: batch/v1
kind: Job
metadata:
  name: es-config-job
spec:
  parallelism: 1
  completions: 1
  template:
    metadata:
      name: es-config-job
    spec:
      restartPolicy: Never
      initContainers:
        - name: wait-for-elasticearch
          image: alpine
          command:
            [
              "sh",
              "-c",
              "for i in $(seq 1 300); do nc -zvw1 es-es-http 9200 && exit 0 || sleep 3; done; exit 1",
            ]
      containers:
        - name: es-config-job
          image: appropriate/curl
          env:
            - name: ELASTICSEARCH_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: es-es-elastic-user
                  key: elastic
          command:
            - /bin/sh
            - -c
            - |
              curl -v -k -u elastic:$ELASTICSEARCH_PASSWORD -X PUT https://es-es-http:9200/_ilm/policy/logstash-ilm-policy?pretty -H 'Content-Type: application/json' -d '
              {
                "policy": {
                  "phases": {
                    "hot": {
                      "actions": {
                        "rollover": {
                          "max_age": "3d"
                        }
                      }
                    },
                    "delete": {
                      "min_age": "0s",
                      "actions": {
                        "delete": {}
                      }
                    }
                  }
                }
              }
              '
              curl -v -k -u elastic:$ELASTICSEARCH_PASSWORD -X PUT https://es-es-http:9200/_template/logstash_index_template?pretty -H 'Content-Type: application/json' -d '
              {
                "index_patterns": ["logstash-*"],
                "settings": {
                  "number_of_shards": 1,
                  "number_of_replicas": 1,
                  "index.lifecycle.name": "logstash-ilm-policy",
                  "index.lifecycle.rollover_alias": "logstash"
                }
              }
              '