Minikube: dashboard on none: unexpected response code: 503
I cannot get minikube dashboard to work. Possible related issue #2380.
Minikube was started with these options:
sudo -E minikube start --vm-driver=none --extra-config=kubelet.resolv-conf=/run/systemd/resolve/resolv.conf --extra-config=kubelet.cgroup-driver=systemd
``` bash
kube@kube:~$ minikube version
minikube version: v1.1.0
``` bash
kube@kube:~$ kubectl get pods -n kube-system kubernetes-dashboard-5f7b999d65-jmv67
NAME READY STATUS RESTARTS AGE
kubernetes-dashboard-5f7b999d65-jmv67 1/1 Running 1 15d
``` bash
kube@kube:~$ kubectl get svc -n kube-system kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard ClusterIP 10.97.218.152
``` bash
kube@kube:~$ kubectl get deployments -n kube-system kubernetes-dashboard
NAME READY UP-TO-DATE AVAILABLE AGE
kubernetes-dashboard 1/1 1 1 15d
``` bash
kube@kube:~$ sudo -E minikube dashboard --logtostderr --v=2
I0524 19:19:50.398017 11405 none.go:231] checking for running kubelet ...
I0524 19:19:50.398044 11405 exec_runner.go:39] Run: systemctl is-active --quiet service kubelet
- Enabling dashboard ...
I0524 19:19:50.404894 11405 none.go:231] checking for running kubelet ...
I0524 19:19:50.404906 11405 exec_runner.go:39] Run: systemctl is-active --quiet service kubelet - Verifying dashboard health ...
I0524 19:19:50.448181 11405 service.go:225] Found service: &Service{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:kubernetes-dashboard,GenerateName:,Namespace:kube-system,SelfLink:/api/v1/namespaces/kube-system/services/kubernetes-dashboard,UID:75c9cad3-7259-11e9-9cee-9829a6fadae6,ResourceVersion:1674809,Generation:0,CreationTimestamp:2019-05-09 12:53:36 +0000 UTC,DeletionTimestamp:,DeletionGracePeriodSeconds:nil,Labels:map[string]string{addonmanager.kubernetes.io/mode: Reconcile,app: kubernetes-dashboard,kubernetes.io/minikube-addons: dashboard,kubernetes.io/minikube-addons-endpoint: dashboard,},Annotations:map[string]string{kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","app":"kubernetes-dashboard","kubernetes.io/minikube-addons":"dashboard","kubernetes.io/minikube-addons-endpoint":"dashboard"},"name":"kubernetes-dashboard","namespace":"kube-system"},"spec":{"ports":[{"port":80,"targetPort":9090}],"selector":{"app":"kubernetes-dashboard"}}}
,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:ServiceSpec{Ports:[{ TCP 80 {0 9090 } 0}],Selector:map[string]string{app: kubernetes-dashboard,},ClusterIP:10.97.218.152,Type:ClusterIP,ExternalIPs:[],SessionAffinity:None,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:nil,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[],},},} - Launching proxy ...
I0524 19:19:50.448629 11405 dashboard.go:139] Executing: /usr/local/bin/kubectl [/usr/local/bin/kubectl --context minikube proxy --port=0]
I0524 19:19:50.449049 11405 dashboard.go:144] Waiting for kubectl to output host:port ...
I0524 19:19:50.506108 11405 dashboard.go:162] proxy stdout: Starting to serve on 127.0.0.1:40167 - Verifying proxy health ...
I0524 19:19:50.524408 11405 dashboard.go:199] http://127.0.0.1:40167/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ response:&{Status:503 Service Unavailable StatusCode:503 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Content-Length:[182] Content-Type:[application/json] Date:[Fri, 24 May 2019 19:19:50 GMT]] Body:0xc0004b9600 ContentLength:182 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000406400 TLS: }
I0524 19:19:50.524546 11405 utils.go:125] error: Temporary Error: unexpected response code: 503 - sleeping 1s
I0524 19:19:51.524706 11405 utils.go:114] retry loop 1
I0524 19:19:51.528851 11405 dashboard.go:199] http://127.0.0.1:40167/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ response:&{Status:503 Service Unavailable StatusCode:503 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Content-Length:[182] Content-Type:[application/json] Date:[Fri, 24 May 2019 19:19:51 GMT]] Body:0xc0002e4680 ContentLength:182 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc00011d600 TLS: }
I0524 19:19:51.528977 11405 utils.go:125] error: Temporary Error: unexpected response code: 503 - sleeping 1s
I0524 19:19:52.529131 11405 utils.go:114] retry loop 2
I0524 19:19:52.534004 11405 dashboard.go:199] http://127.0.0.1:40167/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ response:&{Status:503 Service Unavailable StatusCode:503 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Content-Length:[182] Content-Type:[application/json] Date:[Fri, 24 May 2019 19:19:52 GMT]] Body:0xc0002e4740 ContentLength:182 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc00011dd00 TLS: }
I0524 19:19:52.534103 11405 utils.go:125] error: Temporary Error: unexpected response code: 503 - sleeping 1s
I0524 19:19:53.534307 11405 utils.go:114] retry loop 3
I0524 19:19:53.539371 11405 dashboard.go:199] http://127.0.0.1:40167/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ response:&{Status:503 Service Unavailable StatusCode:503 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Content-Length:[182] Content-Type:[application/json] Date:[Fri, 24 May 2019 19:19:53 GMT]] Body:0xc0004b9900 ContentLength:182 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc00011de00 TLS: }
I0524 19:19:53.539473 11405 utils.go:125] error: Temporary Error: unexpected response code: 503 - sleeping 1s
``` bash
kube@kube:~$ sudo -E minikube dashboard --url
* Enabling dashboard ...
* Verifying dashboard health ...
* Launching proxy ...
* Verifying proxy health ...
X http://127.0.0.1:34183/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ is not responding properly: Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
...
kube@kube:~$
OS: CentOS Linux release 7.6.1810 (Core)
irrgit
👍9
😕1
All 10 comments
Hi
I am encounter the same problem with --vm-driver=none
CentOS 7 minimal install
root@minikube[/tmp/Minikube]# minikube dashboard
* Enabling dashboard ...
* Verifying dashboard health ...
* Launching proxy ...
* Verifying proxy health ...
X http://127.0.0.1:33959/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ is not responding properly: Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
...
root@minikube[/tmp/Minikube]# minikube version
minikube version: v1.1.0
winds6206
on 27 May 2019
👍6
+1
Ubuntu 18.04
$ minikube version
minikube version: v1.1.0
$ sudo minikube start --image-mirror-country=cn
......
$ sudo minikube dashboard
馃攲 Enabling dashboard ...
馃 Verifying dashboard health ...
馃殌 Launching proxy ...
馃 Verifying proxy health ...
馃挘 http://127.0.0.1:40339/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ is not responding properly: Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
......
lianghuiyuan
on 27 May 2019
👍4
I had the same issue
But managed to make it work using --docker-env parameters and setting HTTP_PROXY, HTTPS_PROXY and NO_PROXY env variables as proposed here
minikube stop
minikube delete
rm -rf ~/.minikube
minikube start --vm-driver=none --docker-env HTTP_PROXY=http://myproxy.com:8080 --docker-env HTTPS_PROXY=http://myproxy.com:8080 --docker-env NO_PROXY=127.0.0.1
Ubuntu 16.04
minikube 1.1.0
NovikovEvgeny
on 27 May 2019
👍2
Also having the same issue on CentOS 7.6 with minikube 1.1.0:
minikube start --vm-driver=none --extra-config=apiserver.authorization-mode=RBAC
minikube dashboard
馃攲 Enabling dashboard ...
馃 Verifying dashboard health ...
馃殌 Launching proxy ...
馃 Verifying proxy health ...
馃挘 http://127.0.0.1:36410/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ is not responding properly: Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
Temporary Error: unexpected response code: 503
UPDATE
Managed to solve this by following suggest in this link
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
zosocanuck
on 31 May 2019
👍6
Here are the logs for the pod of the dashboard failing. I guess after trying to enable it too many times, it has entered a failed state now.
kube@kube:~$ kubectl logs -f kubernetes-dashboard-79dd6bfc48-rbll6 -n kube-system
2019/06/05 20:20:33 Starting overwatch
2019/06/05 20:20:33 Using in-cluster config to connect to apiserver
2019/06/05 20:20:33 Using service account token for csrf signing
2019/06/05 20:20:33 Successful initial request to the apiserver, version: v1.14.2
2019/06/05 20:20:33 Generating JWE encryption key
2019/06/05 20:20:33 New synchronizer has been registered: kubernetes-dashboard-key-holder-kube-system. Starting
2019/06/05 20:20:33 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kube-system
2019/06/05 20:20:33 Synchronizer kubernetes-dashboard-key-holder-kube-system exited with error: unexpected object: &Secret{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Data:map[string][]byte{},Type:,StringData:map[string]string{},}
2019/06/05 20:20:33 Storing encryption key in a secret
panic: secrets is forbidden: User "system:serviceaccount:kube-system:default" cannot create resource "secrets" in API group "" in the namespace "kube-system"
goroutine 1 [running]:
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.(*rsaKeyHolder).init(0xc42040a2e0)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:131 +0x35e
github.com/kubernetes/dashboard/src/app/backend/auth/jwe.NewRSAKeyHolder(0x1367500, 0xc42046e180, 0xc42046e180, 0x1213a6e)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/auth/jwe/keyholder.go:170 +0x64
main.initAuthManager(0x13663e0, 0xc420520120, 0xc4204dbcd8, 0x1)
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:185 +0x12c
main.main()
/home/travis/build/kubernetes/dashboard/.tmp/backend/src/github.com/kubernetes/dashboard/src/app/backend/dashboard.go:103 +0x26b
I fixed issue deleting .kube and .minikube directories from my home , maybe it happens because I previously testing using virtualbox let some problems there.
try it ;)
anibalardid
on 19 Jun 2019
👍2
I fixed issue .
just:
minikube stop
minikube start
renhe134880
on 1 Jul 2019
👍7
resolved follow Minikube RBAC Quick Start
https://gist.github.com/F21/08bfc2e3592bed1e931ec40b8d2ab6f5
cc189
on 15 Aug 2019
❤1
@irrgit - I believe this issue is now addressed by minikube v1.4, as it upgrades the dashboard to 2.0 and uses a different RBAC configuration. If you still see this issue with minikube v1.4 or higher, please reopen this issue by commenting with /reopen
Thank you for reporting this issue!
tstromberg
on 20 Sep 2019
/reopen
when i run it sometimes hangs forever and sometimes throws 503
garretsidzaka@cloudstack:/usr/bin$ minikube version
minikube version: v1.4.0
commit: 7969c25a98a018b94ea87d949350f3271e9d64b6
ubuntu vm 1804
vm-driver=none
garretsidzaka@cloudstack:~$ sudo minikube dashboard
- Verifying dashboard health ...
- Launching proxy ...
- Verifying proxy health ...
^C
garretsidzaka@cloudstack:~$
GarretSidzaka
on 31 Oct 2019
👍2
Was this page helpful?
0 / 5 - 0 ratings
Related issues
alber70g
路
3Comments
mrtkp9993
路
3Comments
hank-cp
路
3Comments
wudongyin
路
3Comments
RaeesBhatti
路
3Comments
Most helpful comment
I fixed issue .
just:
minikube stop
minikube start