Dashboard: TLS handshake error: first record does not look like a TLS handshake

[r-divakaran-hrs]

Environment

Dashboard version: 0.6.1
Kubernetes version: v1.8.6
Operating system: ubuntu
Node.js version:
Go version:

Steps to reproduce

I have installed kubernetes-dashboard using helm. The command used was

helm install --name kubernetes-dashboard stable/kubernetes-dashboard --namespace kube-system --set rbac.create=false --set ingress.enabled=true

But then when am accessing it using the ingress hostname its giving and SSL related error. As per pod logs

2018/03/16 14:58:41 http: TLS handshake error from 100.96.3.7:56916: tls: first record does not look like a TLS handshake

The ingress looks like

metadata:
  annotations:
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/sign_in
    nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
  creationTimestamp: 2018-03-16T14:16:48Z
  generation: 3
  labels:
    app: kubernetes-dashboard
    chart: kubernetes-dashboard-0.6.1
    heritage: Tiller
    release: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
  resourceVersion: "4072579"
  selfLink: /apis/extensions/v1beta1/namespaces/kube-system/ingresses/kubernetes-dashboard
  uid: aa54f4f0-2924-11e8-aedc-0a717f21e61e
spec:
  rules:
  - host: dashboard.atkubernetes-lab.com
    http:
      paths:
      - backend:
          serviceName: kubernetes-dashboard
          servicePort: 443
        path: /
  tls:
  - hosts:
    - dashboard.atkubernetes-lab.com
    secretName: dashboard-tls

Observed result

I am able to access the dashboard via API and token, but just the via the ingress resource it's not and ending up in error as I mentioned above.

Expected result

I am expecting to have the dashboard rendering properly when access via dashboard.atkubernetes-lab.com.

Comments

It could be a config issue on my side, but any insights would be of great help. I am using kubernetes-dashboard on few of my other clusters, which are installed using helm. But they are not on the latest version, in the sense its using the image before 443 port was enforced.

Thanks

[floreks]

We can't really help with advanced configurations. Maybe there is an issue on helm side or on your side. It requires access to the cluster anyway to diagnose the config issue.

[r-divakaran-hrs]

Can you tell me how would I be able to install dashboard chart version 0.5.3 !?

[floreks]

Try to ask for help on helm repository. We can help with Dashboard related issues only. It is configuration issue.

[tonglv]

[root@kubernetes-110 dashboard]# kubectl log -f pod/kubernetes-dashboard-545d8dc58-kf6rp -n kube-system
log is DEPRECATED and will be removed in a future version. Use logs instead.
2018/12/13 08:04:22 Using in-cluster config to connect to apiserver
2018/12/13 08:04:22 Using service account token for csrf signing
2018/12/13 08:04:22 No request provided. Skipping authorization
2018/12/13 08:04:22 Starting overwatch
2018/12/13 08:04:22 Successful initial request to the apiserver, version: v1.13.0
2018/12/13 08:04:22 Generating JWE encryption key
2018/12/13 08:04:22 New synchronizer has been registered: kubernetes-dashboard-key-holder-kube-system. Starting
2018/12/13 08:04:22 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2018/12/13 08:04:23 Initializing JWE encryption key from synchronized object
2018/12/13 08:04:23 Creating remote Heapster client for http://heapster
2018/12/13 08:04:23 Successful request to heapster
2018/12/13 08:04:23 Auto-generating certificates
2018/12/13 08:04:23 Successfully created certificates
2018/12/13 08:04:23 Serving securely on HTTPS port: 8443
2018/12/13 08:56:38 http: TLS handshake error from 10.180.160.113:6757: tls: first record does not look like a TLS handshake
2018/12/13 08:56:38 http: TLS handshake error from 10.180.160.113:6759: tls: first record does not look like a TLS handshake
2018/12/13 08:56:43 http: TLS handshake error from 10.180.160.113:6787: tls: first record does not look like a TLS handshake
2018/12/14 02:31:09 http: TLS handshake error from 10.180.160.113:45649: tls: first record does not look like a TLS handshake
2018/12/14 02:31:10 http: TLS handshake error from 10.180.160.113:45657: tls: first record does not look like a TLS handshake
2018/12/14 02:31:15 http: TLS handshake error from 10.180.160.113:45681: tls: first record does not look like a TLS handshake
2018/12/14 03:25:39 http: TLS handshake error from 10.180.160.113:60043: tls: first record does not look like a TLS handshake
2018/12/14 03:46:45 http: TLS handshake error from 10.180.160.113:5650: tls: first record does not look like a TLS handshake
2018/12/14 05:46:16 http: TLS handshake error from 10.180.160.113:37216: tls: first record does not look like a TLS handshake
2018/12/14 05:46:36 http: TLS handshake error from 10.180.160.113:37308: tls: first record does not look like a TLS handshake
2018/12/14 05:47:01 http: TLS handshake error from 10.180.160.113:37426: tls: first record does not look like a TLS handshake
2018/12/14 05:47:51 http: TLS handshake error from 10.180.160.113:37648: tls: first record does not look like a TLS handshake
2018/12/14 05:49:11 http: TLS handshake error from 10.180.160.113:38002: tls: first record does not look like a TLS handshake

help me

[sergey-vin]

I found the problem.
Here are 2 nginx configs of ingress controller, the one - from working cluster, the other - from a cluster where dashboard doesn't work:

working:

              proxy_next_upstream_tries               3;           
              proxy_pass https://upstream_balancer;                 
              proxy_redirect                          off;         

broken:

            proxy_next_upstream_tries               3;
            proxy_pass http://upstream_balancer;
            proxy_redirect                          off;

the diff is in http vs https schema.

[thehoneycomb]

try to change the protocol from HTTPS to HTTP after /services/ in the dashboard URL.

E.G:
from
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/HTTPS:kubernetes-dashboard:/proxy/

to
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/HTTP:kubernetes-dashboard:/proxy/.

Best