Jaeger: Can jaeger set ssl_verify false when connect elastic search with ssl?

[johnzheng1975]

I set up elastic search with search guard plug in.

kibana can access successfully with below configure,

  # Set to false to have a complete disregard for the validity of the SSL certificate.
    elasticsearch.ssl.verificationMode: none

Fulentd can access successfully

host 10.200.199.79
port 30920
scheme https
ssl_verify false
user admin
password mypassword
ssl_version TLSv1_2 

However, jaeger always failed like:
{"level":"fatal","ts":1555073973.330164,"caller":"all-in-one/main.go:101","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head https://10.200.199.79:30920: x509: certificate is valid for 127.0.0.1, not 10.200.199.79: no Elasticsearch node available"

[johnzheng1975]

Here is jaeger deploy

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "3"
  creationTimestamp: 2019-03-05T06:50:38Z
  generation: 3
  labels:
    app: jaeger
    jaeger-infra: jaeger-deployment2
  name: jaeger-deployment2
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
      jaeger-infra: jaeger-pod
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: jaeger
        jaeger-infra: jaeger-pod
    spec:
      volumes:
      - name: volcert
        secret:
          secretName: certes
      containers:
      - env:
        - name: COLLECTOR_ZIPKIN_HTTP_PORT
          value: "9411"
        - name: SPAN_STORAGE_TYPE
          value: elasticsearch
        - name: ES_SERVER_URLS
          value: https://10.200.199.79:30920
        - name: ES_INDEX_PREFIX
          value: dev1ctmp
        - name: ES_USERNAME
          value: admin
        - name: ES_PASSWORD
          value: adminP
        - name: ES_TLS_CA
          value: /tmp/es-root-ca.pem
        image: jaegertracing/all-in-one
        volumeMounts:
        - name: volcert 
          readOnly: true
          mountPath: "/tmp/"
        imagePullPolicy: Always
        name: jaeger
        ports:
        - containerPort: 5775
          protocol: UDP
        - containerPort: 6831
          protocol: UDP
        - containerPort: 6832
          protocol: UDP
        - containerPort: 16686
          protocol: TCP
        - containerPort: 9411
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 16686
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            memory: 2Gi
          requests:
            memory: 200Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

Here is cm of key:

kubectl create secret generic certes --from-file=es-root-ca.pem -n kube-system

[yurishkuro]

It looks like tls.Config type has a field InsecureSkipVerify. We can expose it as one of the --es.tls.* CLI flags.

Are you interested in submitting a PR?

[johnzheng1975]

Thanks you are trying to fix this issue.

Just a confirm: which parameter should I set in k8s config later, as below?
- name: ES_TLS_SKIP_HOST_VERIFY
value: TRUE

[jordo1138]

It looks like tls.Config type has a field InsecureSkipVerify. We can expose it as one of the --es.tls.* CLI flags.

Are you interested in submitting a PR?

was almost finished with the exact same changes in my fork and going to work on the tests when I saw this issue...Will be a good feature as often there may be reasons to have different host/cert on some es node you want to connect to in test
Thanks! @stefanvassilev

[stefanvassilev]

@jordo1138 what kind of tests do you have in mind? I have added a unit test case in plugin/storage/es/options_test.go

[jordo1138]

@stefanvassilev
your test looks good, was referring in my own test env I often have a es node that has no matching ssl cert bc it’s ip based and has changed etc. there’s other workarounds on both side but it’s a nice option to have when needed. Thanks