Dashboard version: v1.8.0
Kubernetes version: v1.8.2
Operating system: Centos7
Node.js version:
Go version: go version go1.8.3 linux/amd64
Ran the following :
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
Pod logs
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2017/12/14 18:20:41 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/14 18:20:41 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2017/12/14 18:20:41 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/14 18:20:41 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2017/12/14 18:20:41 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/14 18:20:41 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2017/12/14 18:20:41 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/14 18:20:41 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
2017/12/14 18:20:41 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/14 18:20:41 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: kubernetes-dashboard-key-holder-kube-system watch ended with timeout
2017/12/14 18:20:41 Restarting synchronizer: kubernetes-dashboard-key-holder-kube-system.
panic: secrets is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot create secrets in the namespace "kube-system"
goroutine 1 [running]:
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.(*rsaKeyHolder).init(0xc42025bfc0)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:132 +0x2d3
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.NewRSAKeyHolder(0x1a78da0, 0xc4201d5260, 0xc4201d5260, 0x1278920)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:171 +0x83
main.initAuthManager(0x1a77300, 0xc420067a40, 0x384, 0x1, 0x1)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:160 +0x12f
main.main()
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:94 +0x27b
describing pod
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 5m default-scheduler Successfully assigned kubernetes-dashboard-7486b894c6-8phqr to master-3
Normal SuccessfulMountVolume 5m kubelet, master-3 MountVolume.SetUp succeeded for volume "tmp-volume"
Normal SuccessfulMountVolume 5m kubelet, master-3 MountVolume.SetUp succeeded for volume "kubernetes-dashboard-certs"
Normal SuccessfulMountVolume 5m kubelet, master-3 MountVolume.SetUp succeeded for volume "kubernetes-dashboard-token-c4w59"
Warning BackOff 5m (x5 over 5m) kubelet, master-3 Back-off restarting failed container
Normal Pulled 5m (x4 over 5m) kubelet, master-3 Container image "gcr.io/google_containers/kubernetes-dashboard-amd64:v1.8.0" already present on machine
Normal Created 5m (x4 over 5m) kubelet, master-3 Created container
Normal Started 5m (x4 over 5m) kubelet, master-3 Started container
Warning FailedSync 45s (x28 over 5m) kubelet, master-3 Error syncing pod
Expected to work.
Service account has been created
$ kubectl get serviceaccounts -n kube-system
NAME SECRETS AGE
default 1 13d
flannel 1 13d
kube-dns 1 13d
kubernetes-dashboard 1 7m
tiller 1 12d
However secrets contain no data?
$ kubectl describe secret kubernetes-dashboard-certs -n kube-system
Name: kubernetes-dashboard-certs
Namespace: kube-system
Labels: k8s-app=kubernetes-dashboard
Annotations:
Type: Opaque
Data
====
Seems to have the create secrets permission
$ kubectl describe role kubernetes-dashboard-minimal -n kube-system
Name: kubernetes-dashboard-minimal
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"annotations":{},"name":"kubernetes-dashboard-minimal","namespace":"kube-system"...
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
configmaps [] [] [create]
configmaps [] [kubernetes-dashboard-settings] [get update]
secrets [] [] [create]
secrets [] [kubernetes-dashboard-certs] [get update delete]
secrets [] [kubernetes-dashboard-key-holder] [get update delete]
services [] [heapster] [proxy]
services/proxy [] [heapster] [get]
services/proxy [] [http:heapster:] [get]
services/proxy [] [https:heapster:] [get]
At a first glance, it looks like some cluster config issue.
Is is ok that secret with certs is empty. Certificates are auto-generated during start. It can be used to override this behavior and allow you to use custom certificates.
How are you provisioning your cluster? Role and SA look correct. Either RBACs are not respected correctly or maybe RoleBinding was not created.
If I understand correctly:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
I checked the ClusterRoleBindings and there is no kubernetes-dashboard-minimal :
$ kubectl get clusterrolebindings -n kube-system
NAME AGE
add-on-cluster-admin 13d
cluster-admin 14d
flannel 14d
prometheus 1d
prometheus-operator 1d
rook-agent 14d
rook-operator 3d
system:basic-user 14d
system:controller:attachdetach-controller 14d
system:controller:certificate-controller 14d
system:controller:cronjob-controller 14d
system:controller:daemon-set-controller 14d
system:controller:deployment-controller 14d
system:controller:disruption-controller 14d
system:controller:endpoint-controller 14d
system:controller:generic-garbage-collector 14d
system:controller:horizontal-pod-autoscaler 14d
system:controller:job-controller 14d
system:controller:namespace-controller 14d
system:controller:node-controller 14d
system:controller:persistent-volume-binder 14d
system:controller:pod-garbage-collector 14d
system:controller:replicaset-controller 14d
system:controller:replication-controller 14d
system:controller:resourcequota-controller 14d
system:controller:route-controller 14d
system:controller:service-account-controller 14d
system:controller:service-controller 14d
system:controller:statefulset-controller 14d
system:controller:ttl-controller 14d
system:discovery 14d
system:kube-apiserver 14d
system:kube-controller-manager 14d
system:kube-dns 14d
system:kube-scheduler 14d
system:node 14d
system:node-proxier 14d
tiller 13d
$ kubectl get clusterrole -n kube-system
NAME AGE
admin 14d
cluster-admin 14d
edit 14d
flannel 14d
prometheus 1d
prometheus-operator 1d
rook-agent 14d
rook-operator 3d
system:auth-delegator 14d
system:basic-user 14d
system:certificates.k8s.io:certificatesigningrequests:nodeclient 14d
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 14d
system:controller:attachdetach-controller 14d
system:controller:certificate-controller 14d
system:controller:cronjob-controller 14d
system:controller:daemon-set-controller 14d
system:controller:deployment-controller 14d
system:controller:disruption-controller 14d
system:controller:endpoint-controller 14d
system:controller:generic-garbage-collector 14d
system:controller:horizontal-pod-autoscaler 14d
system:controller:job-controller 14d
system:controller:namespace-controller 14d
system:controller:node-controller 14d
system:controller:persistent-volume-binder 14d
system:controller:pod-garbage-collector 14d
system:controller:replicaset-controller 14d
system:controller:replication-controller 14d
system:controller:resourcequota-controller 14d
system:controller:route-controller 14d
system:controller:service-account-controller 14d
system:controller:service-controller 14d
system:controller:statefulset-controller 14d
system:controller:ttl-controller 14d
system:discovery 14d
system:heapster 14d
system:kube-aggregator 14d
system:kube-apiserver-to-kubelet 14d
system:kube-controller-manager 14d
system:kube-dns 14d
system:kube-scheduler 14d
system:node 14d
system:node-bootstrapper 14d
system:node-problem-detector 14d
system:node-proxier 14d
system:persistent-volume-provisioner 14d
view 14d
I deleted and attempted to recreate the clusterrole in a new yaml:
$ kubectl apply -f rolebinding.yaml
rolebinding "kubernetes-dashboard-minimal" configured
And it still doesn't get created. What could be my issue here?
Also:
$ kubectl apply -f rolebinding.yaml -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"rbac.authorization.k8s.io/v1","kind":"RoleBinding","metadata":{"annotations":{},"name":"kubernetes-dashboard-minimal","namespace":"kube-system"},"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubernetes-dashboard-minimal"},"subjects":[{"kind":"ServiceAccount","name":"kubernetes-dashboard","namespace":"kube-system"}]}
creationTimestamp: 2017-12-14T18:17:47Z
name: kubernetes-dashboard-minimal
namespace: kube-system
resourceVersion: "1955084"
selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings/kubernetes-dashboard-minimal
uid: 169a2a1c-e0fb-11e7-9ec2-fa163e0a0558
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
$ kubectl get roles -n kube-system
NAME AGE
extension-apiserver-authentication-reader 14d
kubernetes-dashboard-minimal 19h
system::leader-locking-kube-controller-manager 14d
system::leader-locking-kube-scheduler 14d
system:controller:bootstrap-signer 14d
system:controller:cloud-provider 14d
system:controller:token-cleaner 14d
Check for existence of RoleBinding not ClusterRoleBinding in kube-system namespace.
$ kubectl get RoleBinding -n kube-system
NAME AGE
kubernetes-dashboard-minimal 20h
system::leader-locking-kube-controller-manager 14d
system::leader-locking-kube-scheduler 14d
system:controller:bootstrap-signer 14d
system:controller:cloud-provider 14d
system:controller:token-cleaner 14d
$ kubectl describe RoleBinding kubernetes-dashboard-minimal -n kube-system
Name: kubernetes-dashboard-minimal
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"rbac.authorization.k8s.io/v1","kind":"RoleBinding","metadata":{"annotations":{},"name":"kubernetes-dashboard-minimal","namespace":"kube-...
Role:
Kind: Role
Name: kubernetes-dashboard-minimal
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount kubernetes-dashboard kube-system
All resources are there. Everything looks good, yet apiserver still does not allow Dashboard to create a secret. Looks like some core issue to me. Can you update your cluster to the latest version? How did you create the cluster (i.e. kubeadm)?
No kubeadm. kubernetes the hard way using an ansible playbook. kubeadm does not work in my environment.
Can you try to remove Dashboard and change in our yaml rbac.authorization.k8s.io/v1 to rbac.authorization.k8s.io/v1beta1, and then redeploy? I'd also try updating to latest K8S version.
So, I did as you suggested upgraded the k8s cluster (rbac.authorization.k8s.io/v1beta1 didn't work)
$ kubectl version --short
Client Version: v1.9.0-beta.2
Server Version: v1.9.0-beta.2
Played around with a few things and managed to get it "working"; I've exposed my service /w a NodePort (temporarily for tests) and used nginx as a reverse proxy to access the dashboard.
The dashboard does absolutely nothing when presented with my kubeconfig and nothing either with a bearer token - no error, just stays there.
2017/12/15 19:47:13 [2017-12-15T19:47:13Z] Outcoming response to 10.244.5.0:43956 with 500 status code
2017/12/15 19:47:15 Getting application global configuration
2017/12/15 19:47:15 Application configuration {"serverTime":1513367235196}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Incoming HTTP/1.1 GET /api/v1/settings/global request from 10.244.5.0:43962: {}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Outcoming response to 10.244.5.0:43962 with 200 status code
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Incoming HTTP/1.1 GET /api/v1/systembanner request from 10.244.5.0:43964: {}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Outcoming response to 10.244.5.0:43964 with 200 status code
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:43966: {}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Outcoming response to 10.244.5.0:43966 with 200 status code
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Incoming HTTP/1.1 GET /api/v1/rbac/status request from 10.244.5.0:43968: {}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Outcoming response to 10.244.5.0:43968 with 200 status code
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Incoming HTTP/1.1 GET /api/v1/login/modes request from 10.244.5.0:43970: {}
2017/12/15 19:47:15 [2017-12-15T19:47:15Z] Outcoming response to 10.244.5.0:43970 with 200 status code
2017/12/15 19:47:31 [2017-12-15T19:47:31Z] Incoming HTTP/1.1 GET /api/v1/csrftoken/login request from 10.244.5.0:43984: {}
2017/12/15 19:47:31 [2017-12-15T19:47:31Z] Outcoming response to 10.244.5.0:43984 with 200 status code
2017/12/15 19:47:31 [2017-12-15T19:47:31Z] Incoming HTTP/1.1 POST /api/v1/login request from 10.244.5.0:43986: {
"kubeConfig": "",
"password": "",
"token": "......",
19
2017/12/15 19:48:58 [2017-12-15T19:48:58Z] Outcoming response to 10.244.5.0:44060 with 200 status code
2017/12/15 19:48:58 [2017-12-15T19:48:58Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:44062: {}
2017/12/15 19:48:58 [2017-12-15T19:48:58Z] Outcoming response to 10.244.5.0:44062 with 200 status code
2017/12/15 19:49:02 [2017-12-15T19:49:02Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:44064: {}
2017/12/15 19:49:02 [2017-12-15T19:49:02Z] Outcoming response to 10.244.5.0:44064 with 200 status code
2017/12/15 19:49:03 [2017-12-15T19:49:03Z] Incoming HTTP/1.1 GET /api/v1/overview?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:44066: {}
2017/12/15 19:49:03 Getting config category
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list persistentvolumeclaims at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: configmaps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list configmaps at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: secrets is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list secrets at the cluster scope
2017/12/15 19:49:03 Getting discovery and load balancing category
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: services is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list services at the cluster scope
If i enter an invalid token, I get an error message back.
Authentication failed. Please try again.
If i "skip"
2017/12/15 19:49:03 Getting lists of all workloads
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: jobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list jobs.batch at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list statefulsets.apps at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 19:49:03 No metric client provided. Skipping metrics.
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: deployments.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list deployments.apps at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps at the cluster scope
2017/12/15 19:49:03 No metric client provided. Skipping metrics.
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: cronjobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list cronjobs.batch at the cluster scope
2017/12/15 19:49:03 No metric client provided. Skipping metrics.
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicationcontrollers at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 19:49:03 No metric client provided. Skipping metrics.
2017/12/15 19:49:03 Non-critical error occurred during resource retrieval: daemonsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list daemonsets.apps at the cluster scope
I get those "forbidden" errors.
I also tried using the alternate version (without https) exact same situation;
2017/12/15 20:29:08 Starting overwatch
2017/12/15 20:29:08 Using in-cluster config to connect to apiserver
2017/12/15 20:29:08 Using service account token for csrf signing
2017/12/15 20:29:08 No request provided. Skipping authorization
2017/12/15 20:29:08 Successful initial request to the apiserver, version: v1.9.0-beta.2
2017/12/15 20:29:08 Generating JWE encryption key
2017/12/15 20:29:08 New synchronizer has been registered: kubernetes-dashboard-key-holder-kube-system. Starting
2017/12/15 20:29:08 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2017/12/15 20:29:09 Initializing JWE encryption key from synchronized object
2017/12/15 20:29:09 Creating in-cluster Heapster client
2017/12/15 20:29:09 Serving insecurely on HTTP port: 9090
2017/12/15 20:29:09 Successful request to heapster
2017/12/15 20:31:19 Getting application global configuration
2017/12/15 20:31:19 Application configuration {"serverTime":1513369879224}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Incoming HTTP/1.1 GET /api/v1/settings/global request from 10.244.5.0:40408: {}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Outcoming response to 10.244.5.0:40408 with 200 status code
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Incoming HTTP/1.1 GET /api/v1/systembanner request from 10.244.5.0:40410: {}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Outcoming response to 10.244.5.0:40410 with 200 status code
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40412: {}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Outcoming response to 10.244.5.0:40412 with 200 status code
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Incoming HTTP/1.1 GET /api/v1/rbac/status request from 10.244.5.0:40414: {}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Outcoming response to 10.244.5.0:40414 with 200 status code
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40416: {}
2017/12/15 20:31:19 [2017-12-15T20:31:19Z] Outcoming response to 10.244.5.0:40416 with 200 status code
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Incoming HTTP/1.1 GET /api/v1/overview?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:40418: {}
2017/12/15 20:31:20 Getting config category
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: configmaps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list configmaps at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: secrets is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list secrets at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list persistentvolumeclaims at the cluster scope
2017/12/15 20:31:20 Getting discovery and load balancing category
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: services is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list services at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: ingresses.extensions is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list ingresses.extensions at the cluster scope
2017/12/15 20:31:20 Getting lists of all workloads
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicationcontrollers at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: cronjobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list cronjobs.batch at the cluster scope
2017/12/15 20:31:20 Getting pod metrics
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: jobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list jobs.batch at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: deployments.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list deployments.apps at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list statefulsets.apps at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: daemonsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list daemonsets.apps at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods at the cluster scope
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events at the cluster scope
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Outcoming response to 10.244.5.0:40418 with 200 status code
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40420: {}
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Outcoming response to 10.244.5.0:40420 with 200 status code
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Incoming HTTP/1.1 GET /api/v1/overview/default?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:40422: {}
2017/12/15 20:31:20 Getting config category
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list persistentvolumeclaims in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: configmaps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list configmaps in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: secrets is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list secrets in the namespace "default"
2017/12/15 20:31:20 Getting discovery and load balancing category
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: services is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list services in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: ingresses.extensions is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list ingresses.extensions in the namespace "default"
2017/12/15 20:31:20 Getting lists of all workloads
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 Getting pod metrics
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: cronjobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list cronjobs.batch in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list statefulsets.apps in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: daemonsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list daemonsets.apps in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: jobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list jobs.batch in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: deployments.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list deployments.apps in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicationcontrollers in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list replicasets.apps in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:20 No metric client provided. Skipping metrics.
2017/12/15 20:31:20 [2017-12-15T20:31:20Z] Outcoming response to 10.244.5.0:40422 with 200 status code
2017/12/15 20:31:39 [2017-12-15T20:31:39Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40596: {}
2017/12/15 20:31:39 [2017-12-15T20:31:39Z] Outcoming response to 10.244.5.0:40596 with 200 status code
2017/12/15 20:31:39 [2017-12-15T20:31:39Z] Incoming HTTP/1.1 GET /api/v1/cronjob/default?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:40598: {}
2017/12/15 20:31:39 Getting list of all cron jobs in the cluster
2017/12/15 20:31:39 Non-critical error occurred during resource retrieval: cronjobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list cronjobs.batch in the namespace "default"
2017/12/15 20:31:39 [2017-12-15T20:31:39Z] Outcoming response to 10.244.5.0:40598 with 200 status code
2017/12/15 20:31:41 [2017-12-15T20:31:41Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40600: {}
2017/12/15 20:31:41 [2017-12-15T20:31:41Z] Outcoming response to 10.244.5.0:40600 with 200 status code
2017/12/15 20:31:41 [2017-12-15T20:31:41Z] Incoming HTTP/1.1 GET /api/v1/daemonset/default?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:40602: {}
2017/12/15 20:31:41 Non-critical error occurred during resource retrieval: daemonsets.apps is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list daemonsets.apps in the namespace "default"
2017/12/15 20:31:41 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:41 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:41 [2017-12-15T20:31:41Z] Outcoming response to 10.244.5.0:40602 with 200 status code
2017/12/15 20:31:42 [2017-12-15T20:31:42Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 10.244.5.0:40604: {}
2017/12/15 20:31:42 [2017-12-15T20:31:42Z] Outcoming response to 10.244.5.0:40604 with 200 status code
2017/12/15 20:31:42 [2017-12-15T20:31:42Z] Incoming HTTP/1.1 GET /api/v1/job/default?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 10.244.5.0:40606: {}
2017/12/15 20:31:42 Getting list of all jobs in the cluster
2017/12/15 20:31:42 Non-critical error occurred during resource retrieval: jobs.batch is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list jobs.batch in the namespace "default"
2017/12/15 20:31:42 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list pods in the namespace "default"
2017/12/15 20:31:42 Non-critical error occurred during resource retrieval: events is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list events in the namespace "default"
2017/12/15 20:31:42 [2017-12-15T20:31:42Z] Outcoming response to 10.244.5.0:40606 with 200 status code
Was the reverse proxy using HTTP or HTTPS?
Using Skip is equal to using default privileges of Service Account bound to dashboard pod. By default there are pretty much no privileges, that is why you will see warnings with information that current "user" has no permission to do something.
If you want to grant full admin privileges to Dashboard and be able to see everything while using Skip then check this guide.
I didn't add TLS cert so in essence my nginx proxy was stripping the https.
First output : http -> { dashboard } https
And when I ran the other "alternate" dashboard deploy it was http -> { dashboard } http
I definitely do not want to do that! I'd like the dashboard to actually need me to authenticate.
There is info on our wiki pages that in order to be able to log in you need to access Dashboard over HTTPS. Frontend will not allow to log in if current domain does not use HTTPS.
As for the alternate version everything works as intended. Starting from K8S 1.7 (if I remember correctly) default permissions are very limited. You would need to grant Dashboard manually more permissions, i.e. by creating ClusterRoleBinding with cluster-admin ClusterRole for kubernetes-dashboard SA.
@microwavecausesaids any update here? If this was indeed config issue then I'd like to close.
Closing as stale. Ping us or folks from kubernetes-users channel on slack if you need further assistance with cluster configuration.
I meet this issue too, dashboard version is 1.8.1
how to fix it ?
Please reopen this issue @floreks
Hi guys, I'm trying to lunch kubernetes dashboard on premis (latest versions). As described in doc SA kubernetes-dashboard-minimal has limited permissions. So I tried to use another SA - deployment-controller:
kubectl -n kube-system describe secret deployment-controller-token-pr7b9
When I copy token to the Dashboard and press Sign-in it does nothing.
In the log:
kubectl logs -n kube-system kubernetes-dashboard-5bd6f767c7-lrkcl -f
I see following:
2018/03/30 07:59:57 [2018-03-30T07:59:57Z] Incoming HTTP/2.0 POST /api/v1/login request from 10.32.0.1:44400: {
"kubeConfig": "",
"password": "",
"token":"username": ""
}
Is there are any recent recipe how to use Dashboard remotely with SA?
Is there a solution for this issue ?
I have the same issue - any luck with finding a solution?
Try to upadate kubernetes version to 1.9.1 or above and update your cluster ....
Ok thanks
Sent from IBM Verse
ishan upadhyay --- Re: [kubernetes/dashboard] Unable to run dashboard. (#2681) ---
From:"ishan upadhyay" notifications@github.comTo:"kubernetes/dashboard" dashboard@noreply.github.comCc:"arivatibm" ariv@il.ibm.com, "Comment" comment@noreply.github.comDate:Wed, May 23, 2018 7:13 AMSubject:Re: [kubernetes/dashboard] Unable to run dashboard. (#2681)
Try to upadate kubernetes version to 1.9.1 or above and update your cluster ....
鈥擸ou are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.
For those still struggling with this issue, adding this to my cluster solved it for me:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
Source: https://blog.tekspace.io/kubernetes-dashboard-remote-access/
Edit: I should mention (to anyone who finds this post) that this ClusterRoleBinding will permit the kubernetes-dashboard ServiceAccount full-control over your entire cluster. This should be chosen carefully, after reviewing the Access-Control wiki page posted below. @divyangjp has proposed a much more restricted ServiceAccount below with tighter control on what the account can and can't do.
@senorequeso that works for me too. Thank you for the solution. On the other hand, is that a security concern that one can skip to get in?
@senorequeso Thank you it works :) Although bypassing the login is concerning.
@senorequeso That's a security risk. Anybody who chooses to use the solution by @senorequeso check the page https://github.com/kubernetes/dashboard/wiki/Access-control. In the section Admin Privileges it clearly states that IMPORTANT: Make sure that you know what you are doing before proceeding. Granting admin privileges to Dashboard's Service Account might be a security risk.
@divyangjp, agreed. Can we propose a better configuration?
I found this, but using the ServiceAccount, Role and RoleBindings there has not worked on my own cluster:
https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
@alexvicegrab Here's a lot more restricted configuration then cluster-admin
It creates new ServiceAccount named dashboard-viewer and gives it view permission only.
It can see the resources except secrets and can't edit/update anything.
Is this better?
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-viewer
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: dashboard-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: dashboard-viewer
namespace: kube-system
To get token for logging in the dashboard, use this query
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep dashboard-viewer | awk '{print $1}')
Above comment is for dashboard viewing purpose.
But to actually setup dashboard, use this minimal Role and RoleBinding
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard-minimal-role
namespace: kube-system
rules:
# Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
# Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create"]
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal-rolebinding
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal-role
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
Most helpful comment
For those still struggling with this issue, adding this to my cluster solved it for me:
Source: https://blog.tekspace.io/kubernetes-dashboard-remote-access/
Edit: I should mention (to anyone who finds this post) that this ClusterRoleBinding will permit the kubernetes-dashboard ServiceAccount full-control over your entire cluster. This should be chosen carefully, after reviewing the Access-Control wiki page posted below. @divyangjp has proposed a much more restricted ServiceAccount below with tighter control on what the account can and can't do.