Dashboard: No cert/key created within kubernetes-dashboard-certs secret

[cmoulliard]

Environment

Installation method: `kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml`
Kubernetes version: 1.15.6
Dashboard version: 2.0.0-beta8
Operating system: Centos7

Steps to reproduce

1. kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
2. kubectl describe secret/kubernetes-dashboard-certs -n kubernetes-dashboard

 kc describe secret/kubernetes-dashboard-certs -n kubernetes-dashboard        
Name:         kubernetes-dashboard-certs
Namespace:    kubernetes-dashboard
Labels:       k8s-app=kubernetes-dashboard
Annotations:  
Type:         Opaque

Data
====

1. kc describe secret/kubernetes-dashboard-csrf -n kubernetes-dashboard

Name:         kubernetes-dashboard-csrf
Namespace:    kubernetes-dashboard
Labels:       k8s-app=kubernetes-dashboard
Annotations:  
Type:         Opaque

Data
====
csrf:  256 bytes

1. kc describe secret/kubernetes-dashboard-key-holder -n kubernetes-dashboard

Name:         kubernetes-dashboard-key-holder
Namespace:    kubernetes-dashboard
Labels:       k8s-app=kubernetes-dashboard
Annotations:  
Type:         Opaque

Data
====
priv:  1679 bytes
pub:   459 bytes

Observed result

Log of the dashboard

kc logs kubernetes-dashboard-56c5f95c6b-fsg6n -n kubernetes-dashboard
2019/12/18 19:48:12 Starting overwatch
2019/12/18 19:48:12 Using namespace: kubernetes-dashboard
2019/12/18 19:48:12 Using in-cluster config to connect to apiserver
2019/12/18 19:48:12 Using secret token for csrf signing
2019/12/18 19:48:12 Initializing csrf token from kubernetes-dashboard-csrf secret
2019/12/18 19:48:12 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2019/12/18 19:48:12 Successful initial request to the apiserver, version: v1.15.6
2019/12/18 19:48:12 Generating JWE encryption key
2019/12/18 19:48:12 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2019/12/18 19:48:12 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2019/12/18 19:48:12 Initializing JWE encryption key from synchronized object
2019/12/18 19:48:12 Creating in-cluster Sidecar client
2019/12/18 19:48:12 Successful request to sidecar
2019/12/18 19:48:12 Auto-generating certificates
2019/12/18 19:48:12 Successfully created certificates
2019/12/18 19:48:12 Serving securely on HTTPS port: 8443

Expected result

tls.crt and tls.key should be created as part of the secret

[floreks]

Works as designed. The secret is for the user to upload his custom certificates. Our certificates are autogenerated and stored in memory, not in the secret.

/close

[k8s-ci-robot]

@floreks: Closing this issue.

In response to this:

Works as designed. The secret is for the user to upload his custom certificates. Our certificates are autogenerated and stored in memory, not in the secret.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cmoulliard]

What is strange is that I dont suffer from this problem, if I install the dashboard top of a k8s cluster : 1.14.9 and not 1.15.6 !!

Steps followed

git clone https://github.com/snowdrop/k8s-infra
cd k8s-infra/ansible

# Export the IP address of your docker vm where you would like to install the k8s cluster
IP=116.202.128.93

ansible-playbook playbook/generate_inventory.yml -e ip_address=${IP} -e type=hetzner -e filename=${IP}_host
ansible-playbook -i inventory/${IP}_host playbook/post_installation.yml --tags k8s_cluster
ansible-playbook -i inventory/${IP}_host playbook/post_installation.yml --tags ingress
ansible-playbook -i inventory/${IP}_host playbook/post_installation.yml --tags k8s_dashboard -e k8s_dashboard_token_public="<PUBLIC_TOKEN>" -e k8s_dashboard_token_secret="<SECRET_TOKEN>"

Even if the ingress controller cant get the kubernetes-dashboard-certs secrert, then it will generate a fake ssl certificate and with 1.14.9 we can access the console

W1219 09:01:49.523201       8 backend_ssl.go:48] Error obtaining X.509 certificate: secret "kubernetes-dashboard/kubernetes-dashboard-certs" contains no keypair or CA certificate
W1219 09:01:52.849519       8 controller.go:1108] Error getting SSL certificate "kubernetes-dashboard/kubernetes-dashboard-certs": local SSL certificate kubernetes-dashboard/kubernetes-dashboard-certs was not found. Using default certificate
I1219 09:01:52.849560       8 controller.go:172] Configuration changes detected, backend reload required.
I1219 09:01:52.905087       8 controller.go:190] Backend successfully reloaded.
[19/Dec/2019:09:01:52 +0000]TCP200000.000
62.235.14.246 - [62.235.14.246] - - [19/Dec/2019:09:02:22 +0000] "GET / HTTP/2.0" 200 744 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36" 946 0.003 [kubernetes-dashboard-kubernetes-dashboard-443] 10.244.0.95:8443 744 0.002 200 bbc994f1dc4497957f0e10321dff3447
62.235.14.246 - [62.235.14.246] - - [19/Dec/2019:09:02:22 +0000] "GET /runtime.380dd4d7ab4891f91b7b.js HTTP/2.0" 200 2021 "https://k8s-console.116.202.128.93.nip.io/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36" 47 0.003 [kubernetes-dashboard-kubernetes-dashboard-443] 10.244.0.95:8443 2021 0.002 200 26ee437addfcaeaa05b96cf98cae726b

[floreks]

Then this is a Kubernetes related issue, not Dashboard. I have explained how it works on our side.

[cmoulliard]

What do you suggest to do then: open a k8s and/or ingress-controller ticket with the reported issue ? @floreks

[floreks]

I am not entirely sure that this is an issue. It probably works as expected. If you want ingress to pick up the certificates for Dashboard, then upload your own custom certificates to the secret, otherwise, autogenerated secrets are not shared (only stored in memory).

They might have intentionally dropped autogenerating certificates if the related secret is empty. User has to take care of that.

[cmoulliard]

If you want ingress to pick up the certificates for Dashboard, then upload your own custom certificates to the secret, otherwise, autogenerated secrets are not shared (only stored in memory).

Right. In fact, my config is wrong as the content of the secret is empty for kubernetes-dashboard-certs
(= no tls crt and key) but when it is deployed on k8s (= 1.14.x) with ingress nginx controller (= 0.23.0), then a fake SSL Certificate is generated and users can access the URL of the dashboard.