Curator: Env var for http_auth seems to fail to evaluate?

[jordansissel]

A config:

client:
  # ...
  http_auth: ${CURATOR}

Expected Behavior

I believe ${CURATOR} should be evaluated from the env var in the config.

Actual Behavior

2019-08-01 20:08:27,485 INFO      Preparing Action ID: 1, "snapshot"
/usr/local/lib/python3.6/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  return yaml.load(read_file(path))
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/curator/utils.py", line 899, in get_client
    check_version(client)
...
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 178, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.AuthenticationException: AuthenticationException(401, 'security_exception', 'invalid basic authentication header value')

Steps to Reproduce the Problem

set config.http_auth: ${VAR} in config.

Specifications

• Version: Curator 5.7
• Platform: Linux, GKE
• Subsystem:

Context (Environment)

Trying to pass in credentials via env var.

% kubectl describe cronjob curator
    Environment:
      CURATOR:  <set to the key 'auth' in secret 'curator-creds'>  Optional: false
    Mounts:
      /etc/curator/actions from curator-actions (rw)
      /etc/curator/config from curator-config (rw)
  Volumes:
   curator-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      curator-config.yml

% kubectl get -o json configmap curator-config.yml | jq -r '.data["config.yml"]' | egrep 'client:|http_auth:'
client:
  http_auth: ${CURATOR}

% kubectl get -o json secret curator-creds | jq -r .data.auth | base64 -d; echo
curator:redacted

Seems like everything should line up, but it's not evaluating ${CURATOR}

[jordansissel]

invalid basic authentication header value

This error comes from Elasticsearch Security. My hypothesis is that ${CURATOR} is being used literally as the basic auth value. I can reproduce the Elasticsearch response error by doing this:

% curl -H "Authorization: Basic $(echo '${CURATOR}' | base64)" https://my-elasticsearch-hostname/
{"error":{"root_cause":[{"type":"security_exception","reason":"invalid basic authentication header value", ...

[kskewes]

On Kubernetes 1.14 with Curator 5.7.6 and ES 7.3 the following is working for us:

ConfigMap:

  config.yml: |-
    ---
    client:
      hosts:
      - internal-es-http
      port: 9200
      http_auth: ${ELASTIC_CREDS}
...<snip>

Cronjob:

            env:
            - name: ELASTIC_USER
              value: curator
            - name: ELASTIC_PASS
              valueFrom:
                secretKeyRef:
                  name: elastic-curator
                  key: elasticsearch-password
            - name: ELASTIC_CREDS
              value: $(ELASTIC_USER):$(ELASTIC_PASS)

Note Kubernetes will eval/combine the values of previous env vars if you use $() instead of ${}

[jordansissel]

@kskewes ahh, lovely! We'll try this.

(Citation for anyone curious: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#use-configmap-defined-environment-variables-in-pod-commands)

[ajrios33]

We could probably close this issue. Storing credentials in anenv was not working using curator-5.7.0, but infosec upgraded to 5.8.0 and we are seeing the expected behavior.