Microk8s: v1.16.0 stable&candidate break Dashboard Token login 404 redirect error
System
- Ubuntu 18.04
- snap install (today, 2019-09-24):
sudo snap install microk8s --classic --channel=candidate(installs: candidate: v1.16.0 2019-09-21 (918) 187MB classic)
Report tarball:
inspection-report-20190924_151054.tar.gz
Problem
Just started with MicroK8s and following the tutorial at: https://tutorials.ubuntu.com/tutorial/install-a-local-kubernetes-with-microk8s#3
When trying to access the Kubernetes Dashboard, I can successfully access the page, but when I use a Token to log-in, I get a:
error_outline Not Found (404)
the server could not find the requested resource
Redirecting to previous state in 3 seconds...
1 day further of debugging and asking over at the Kubernetes SLACK in channel #microk8s, I found out that this error is only in v1.16 (snap info microk8s):
- candidate: v1.16.0 2019-09-21 (918) 187MB classic
- 1.16/stable: v1.16.0 2019-09-19 (912) 187MB classic
No problem:
- stable: v1.15.3 2019-09-06 (826) 171MB classic
Steps taken
# Previous installation
$ sudo usermod -a -G microk8s ${USER}
$ microk8s.stop
Stopped.
$ snap remove microk8s
microk8s removed
# Just started with MicroK8s. No settings have been changed.
# restart system #
$ snap info microk8s
1.16/stable: v1.16.0 2019-09-19 (912) 187MB classic
$ sudo snap install microk8s --classic --channel=candidate
microk8s (candidate) v1.16.0 from Canonical✓ installed
$ microk8s.enable dns dashboard registry: (click me)
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
Restarting kubelet
DNS is enabled
Applying manifest
secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created
service/monitoring-grafana created
service/monitoring-influxdb created
service/heapster created
deployment.apps/monitoring-influxdb-grafana-v4 created
serviceaccount/heapster created
clusterrolebinding.rbac.authorization.k8s.io/heapster created
configmap/heapster-config created
configmap/eventer-config created
deployment.apps/heapster-v1.5.2 created
If RBAC is not enabled access the dashboard using the default token retrieved with:
token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
microk8s.kubectl -n kube-system describe secret $token
In an RBAC enabled setup (microk8s.enable RBAC) you need to create a user with restricted
permissions as shown in https://github.com/kubernetes/dashboard/wiki/Creating-sample-user
Enabling the private registry
Enabling default storage class
deployment.apps/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
serviceaccount/microk8s-hostpath created
clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
Storage will be available soon
Applying registry manifest
namespace/container-registry created
persistentvolumeclaim/registry-claim created
deployment.apps/registry created
service/registry created
The registry is enabled
$ microk8s.kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
container-registry pod/registry-d7d7c8bc9-bjjds 1/1 Running 0 4m45s
kube-system pod/coredns-9b8997588-pxv26 1/1 Running 0 4m45s
kube-system pod/heapster-v1.5.2-5c58f64f8b-ktst5 4/4 Running 2 4m45s
kube-system pod/hostpath-provisioner-7b9cb5cdb4-4s7mh 1/1 Running 0 4m45s
kube-system pod/kubernetes-dashboard-7c54d59f66-8244p 1/1 Running 0 4m45s
kube-system pod/monitoring-influxdb-grafana-v4-6d599df6bf-4h6pn 2/2 Running 0 4m45s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
container-registry service/registry NodePort 10.152.183.116 <none> 5000:32000/TCP 4m58s
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 5m34s
kube-system service/heapster ClusterIP 10.152.183.34 <none> 80/TCP 4m59s
kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 5m4s
kube-system service/kubernetes-dashboard ClusterIP 10.152.183.46 <none> 443/TCP 4m59s
kube-system service/monitoring-grafana ClusterIP 10.152.183.233 <none> 80/TCP 4m59s
kube-system service/monitoring-influxdb ClusterIP 10.152.183.40 <none> 8083/TCP,8086/TCP 4m59s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
container-registry deployment.apps/registry 1/1 1 1 4m58s
kube-system deployment.apps/coredns 1/1 1 1 5m4s
kube-system deployment.apps/heapster-v1.5.2 1/1 1 1 4m59s
kube-system deployment.apps/hostpath-provisioner 1/1 1 1 4m59s
kube-system deployment.apps/kubernetes-dashboard 1/1 1 1 4m59s
kube-system deployment.apps/monitoring-influxdb-grafana-v4 1/1 1 1 4m59s
NAMESPACE NAME DESIRED CURRENT READY AGE
container-registry replicaset.apps/registry-d7d7c8bc9 1 1 1 4m45s
kube-system replicaset.apps/coredns-9b8997588 1 1 1 4m45s
kube-system replicaset.apps/heapster-v1.5.2-5c58f64f8b 1 1 1 4m45s
kube-system replicaset.apps/hostpath-provisioner-7b9cb5cdb4 1 1 1 4m45s
kube-system replicaset.apps/kubernetes-dashboard-7c54d59f66 1 1 1 4m45s
kube-system replicaset.apps/monitoring-influxdb-grafana-v4-6d599df6bf 1 1 1 4m45s
$ microk8s.kubectl get nodes
NAME STATUS ROLES AGE VERSION
*user* Ready <none> 5m49s v1.16.0
$ microk8s.kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 6m3s
$ microk8s.kubectl cluster-info
Kubernetes master is running at https://127.0.0.1:16443
Heapster is running at https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/heapster/proxy
CoreDNS is running at https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Grafana is running at https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
InfluxDB is running at https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-influxdb:http/proxy
# get token
$ token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
$ microk8s.kubectl -n kube-system describe secret $token
Name: default-token-z64w8
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: 3eea0c67-f3dd-44b3-821a-f670b9244c93
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1107 bytes
namespace: 11 bytes
token: ey*****************************
Browser
Access https://10.152.183.46:443
Firefox browser Console error:
firefox_console-export-2019-9-24_15-37-48.txt
Possibly unhandled rejection: {"data":"MSG_LOGIN_UNAUTHORIZED_ERROR\n","status":401,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"api/v1/rbac/status","headers":{"Accept":"application/json, text/plain, */*"}},"statusText":"Unauthorized","xhrStatus":"complete","resource":{}} vendor.bd425c26.js:6:65383
i https://10.152.183.46/static/vendor.bd425c26.js:6
get https://10.152.183.46/static/vendor.bd425c26.js:6
c https://10.152.183.46/static/vendor.bd425c26.js:6
$digest https://10.152.183.46/static/vendor.bd425c26.js:6
$apply https://10.152.183.46/static/vendor.bd425c26.js:6
l https://10.152.183.46/static/vendor.bd425c26.js:6
v https://10.152.183.46/static/vendor.bd425c26.js:6
t https://10.152.183.46/static/vendor.bd425c26.js:6
Firefox browser Debugger error:
blob://
> /10.152.183.46
>> 5b37132e-83b2-4c15-bb30-7cc318b61d24
Error loading this URI: Protocol error (unknownError): Could not load the source for blob:https://10.152.183.46/5b37132e-83b2-4c15-bb30-7cc318b61d24.
[Exception... "Failed to open input source 'blob:https://10.152.183.46/5b37132e-83b2-4c15-bb30-7cc318b61d24'" nsresult: "0x805303f4 (<unknown>)" location: "JS frame :: resource://devtools/shared/DevToolsUtils.js :: mainThreadFetch/< :: line 666" data: yes]
Stack: mainThreadFetch/<@resource://devtools/shared/DevToolsUtils.js:666:15
mainThreadFetch@resource://devtools/shared/DevToolsUtils.js:531:10
_getSourceText@resource://devtools/server/actors/source.js:299:27
Line: 666, column: 0
Atempted log-in
On Kubernetes Dashboard webpage: Token --> copy token --> sign-in
error_outline Not Found (404)
the server could not find the requested resource
Redirecting to previous state in 3 seconds...
Go to dashboard with different link: https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login
$ microk8s.config
# login with username and password
# Paste token --> Sign in button
# same outcome, get redirected
grafana (works fine)
# grafana: https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
$ microk8s.config
# login with username and password
# works normally
Debug information
$ microk8s.inspect
Inspecting services
Service snap.microk8s.daemon-cluster-agent is running
Service snap.microk8s.daemon-flanneld is running
Service snap.microk8s.daemon-containerd is running
Service snap.microk8s.daemon-apiserver is running
Service snap.microk8s.daemon-apiserver-kicker is running
Service snap.microk8s.daemon-proxy is running
Service snap.microk8s.daemon-kubelet is running
Service snap.microk8s.daemon-scheduler is running
Service snap.microk8s.daemon-controller-manager is running
Service snap.microk8s.daemon-etcd is running
Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
Copy processes list to the final report tarball
Copy snap list to the final report tarball
Copy VM name (or none) to the final report tarball
Copy disk usage information to the final report tarball
Copy memory usage information to the final report tarball
Copy server uptime to the final report tarball
Copy current linux distribution to the final report tarball
Copy openSSL information to the final report tarball
Copy network configuration to the final report tarball
Inspecting kubernetes cluster
Inspect kubernetes cluster
WARNING: IPtables FORWARD policy is DROP. Consider enabling traffic forwarding with: sudo iptables -P FORWARD ACCEPT
The change can be made persistent with: sudo apt-get install iptables-persistent
WARNING: Docker is installed.
Add the following lines to /etc/docker/daemon.json:
{
"insecure-registries" : ["localhost:32000"]
}
and then restart docker with: sudo systemctl restart docker
Building the report tarball
Report tarball is at /var/snap/microk8s/918/inspection-report-20190924_151054.tar.gz
# attached this log
$ sudo iptables -P FORWARD ACCEPT
# Go to https://10.152.183.46/#!/login
# Enter token
# Same problem
What now?
Any more steps I can take to solve this issue?
For now it seems I have to use v1.15.3, but since v.1.16.0 is already in the candidate channel, it seems to me that this is an issue that should not be there anymore.
NumesSanguis
All 3 comments
Thank you for reporting this @NumesSanguis
A fix is on the way to upgrade the dashboard. For now you should be able to deploy the latest dashboard with:
microk8s.kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml
ktsakalozos
on 24 Sep 2019
👍3
@ktsakalozos I'm not sure if your patch is in already in 1.16/candidate, but after the following steps, I get a new dashboard error:
snap remove microk8s
sudo snap install microk8s --classic --channel=1.16/candidate
microk8s.enable dns dashboard registry
Failed to enable dashboard error: (click me)
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
Restarting kubelet
DNS is enabled
Applying manifest
error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"ServiceAccount" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/serviceaccounts/kubernetes-dashboard: http2: server sent GOAWAY and closed the connection; LastStreamID=3, ErrCode=NO_ERROR, debug=""
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "spec":map["ports":[map["port":'\u01bb' "targetPort":'\u20fb']] "selector":map["k8s-app":"kubernetes-dashboard"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "kubernetes-dashboard-certs", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Secret" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-certs" "namespace":"kube-system"] "type":"Opaque"]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/secrets/kubernetes-dashboard-certs: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "kubernetes-dashboard-csrf", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "data":map["csrf":""] "kind":"Secret" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-csrf" "namespace":"kube-system"] "type":"Opaque"]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/secrets/kubernetes-dashboard-csrf: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "kubernetes-dashboard-key-holder", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Secret" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-key-holder" "namespace":"kube-system"] "type":"Opaque"]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/secrets/kubernetes-dashboard-key-holder: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "kubernetes-dashboard-settings", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-settings" "namespace":"kube-system"]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/configmaps/kubernetes-dashboard-settings: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1" "kind":"Role" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "rules":[map["apiGroups":[""] "resourceNames":["kubernetes-dashboard-key-holder" "kubernetes-dashboard-certs" "kubernetes-dashboard-csrf"] "resources":["secrets"] "verbs":["get" "update" "delete"]] map["apiGroups":[""] "resourceNames":["kubernetes-dashboard-settings"] "resources":["configmaps"] "verbs":["get" "update"]] map["apiGroups":[""] "resourceNames":["heapster" "dashboard-metrics-scraper"] "resources":["services"] "verbs":["proxy"]] map["apiGroups":[""] "resourceNames":["heapster" "http:heapster:" "https:heapster:" "dashboard-metrics-scraper" "http:dashboard-metrics-scraper"] "resources":["services/proxy"] "verbs":["get"]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "kubernetes-dashboard", Namespace: ""
Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1" "kind":"ClusterRole" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard"] "rules":[map["apiGroups":["metrics.k8s.io"] "resources":["pods" "nodes"] "verbs":["get" "list" "watch"]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/rbac.authorization.k8s.io/v1/clusterroles/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1" "kind":"RoleBinding" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "roleRef":map["apiGroup":"rbac.authorization.k8s.io" "kind":"Role" "name":"kubernetes-dashboard"] "subjects":[map["kind":"ServiceAccount" "name":"kubernetes-dashboard" "namespace":"kube-system"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "kubernetes-dashboard", Namespace: ""
Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1" "kind":"ClusterRoleBinding" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "name":"kubernetes-dashboard"] "roleRef":map["apiGroup":"rbac.authorization.k8s.io" "kind":"ClusterRole" "name":"kubernetes-dashboard"] "subjects":[map["kind":"ServiceAccount" "name":"kubernetes-dashboard" "namespace":"kube-system"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "spec":map["replicas":'\x01' "revisionHistoryLimit":'\n' "selector":map["matchLabels":map["k8s-app":"kubernetes-dashboard"]] "template":map["metadata":map["labels":map["k8s-app":"kubernetes-dashboard"]] "spec":map["containers":[map["args":["--auto-generate-certificates" "--namespace=kube-system"] "image":"kubernetesui/dashboard:v2.0.0-beta4" "imagePullPolicy":"Always" "livenessProbe":map["httpGet":map["path":"/" "port":'\u20fb' "scheme":"HTTPS"] "initialDelaySeconds":'\x1e' "timeoutSeconds":'\x1e'] "name":"kubernetes-dashboard" "ports":[map["containerPort":'\u20fb' "protocol":"TCP"]] "volumeMounts":[map["mountPath":"/certs" "name":"kubernetes-dashboard-certs"] map["mountPath":"/tmp" "name":"tmp-volume"]]]] "serviceAccountName":"kubernetes-dashboard" "tolerations":[map["effect":"NoSchedule" "key":"node-role.kubernetes.io/master"]] "volumes":[map["name":"kubernetes-dashboard-certs" "secret":map["secretName":"kubernetes-dashboard-certs"]] map["emptyDir":map[] "name":"tmp-volume"]]]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/apps/v1/namespaces/kube-system/deployments/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "dashboard-metrics-scraper", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"dashboard-metrics-scraper"] "name":"dashboard-metrics-scraper" "namespace":"kube-system"] "spec":map["ports":[map["port":'\u1f40' "targetPort":'\u1f40']] "selector":map["k8s-app":"dashboard-metrics-scraper"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/dashboard-metrics-scraper: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "dashboard-metrics-scraper", Namespace: "kube-system"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"dashboard-metrics-scraper"] "name":"dashboard-metrics-scraper" "namespace":"kube-system"] "spec":map["replicas":'\x01' "revisionHistoryLimit":'\n' "selector":map["matchLabels":map["k8s-app":"dashboard-metrics-scraper"]] "template":map["metadata":map["labels":map["k8s-app":"dashboard-metrics-scraper"]] "spec":map["containers":[map["image":"kubernetesui/metrics-scraper:v1.0.1" "livenessProbe":map["httpGet":map["path":"/" "port":'\u1f40' "scheme":"HTTP"] "initialDelaySeconds":'\x1e' "timeoutSeconds":'\x1e'] "name":"dashboard-metrics-scraper" "ports":[map["containerPort":'\u1f40' "protocol":"TCP"]] "volumeMounts":[map["mountPath":"/tmp" "name":"tmp-volume"]]]] "serviceAccountName":"kubernetes-dashboard" "tolerations":[map["effect":"NoSchedule" "key":"node-role.kubernetes.io/master"]] "volumes":[map["emptyDir":map[] "name":"tmp-volume"]]]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/apps/v1/namespaces/kube-system/deployments/dashboard-metrics-scraper: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "monitoring-grafana", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"Grafana"] "name":"monitoring-grafana" "namespace":"kube-system"] "spec":map["ports":[map["port":'P' "protocol":"TCP" "targetPort":"ui"]] "selector":map["k8s-app":"influxGrafana"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-grafana: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "monitoring-influxdb", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"InfluxDB"] "name":"monitoring-influxdb" "namespace":"kube-system"] "spec":map["ports":[map["name":"http" "port":'\u1f93' "targetPort":'\u1f93'] map["name":"api" "port":'\u1f96' "targetPort":'\u1f96']] "selector":map["k8s-app":"influxGrafana"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-influxdb: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "heapster", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"Heapster"] "name":"heapster" "namespace":"kube-system"] "spec":map["ports":[map["port":'P' "targetPort":'\u1f92']] "selector":map["k8s-app":"heapster"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/heapster: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "monitoring-influxdb-grafana-v4", Namespace: "kube-system"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "k8s-app":"influxGrafana" "kubernetes.io/cluster-service":"true" "version":"v4"] "name":"monitoring-influxdb-grafana-v4" "namespace":"kube-system"] "spec":map["replicas":'\x01' "selector":map["matchLabels":map["k8s-app":"influxGrafana" "version":"v4"]] "template":map["metadata":map["annotations":map["scheduler.alpha.kubernetes.io/critical-pod":""] "labels":map["k8s-app":"influxGrafana" "version":"v4"]] "spec":map["containers":[map["image":"k8s.gcr.io/heapster-influxdb-amd64:v1.3.3" "name":"influxdb" "ports":[map["containerPort":'\u1f93' "name":"http"] map["containerPort":'\u1f96' "name":"api"]] "resources":map["limits":map["cpu":"100m" "memory":"500Mi"] "requests":map["cpu":"100m" "memory":"500Mi"]] "volumeMounts":[map["mountPath":"/data" "name":"influxdb-persistent-storage"]]] map["env":[map["name":"INFLUXDB_SERVICE_URL" "value":"http://monitoring-influxdb:8086"] map["name":"GF_AUTH_BASIC_ENABLED" "value":"false"] map["name":"GF_AUTH_ANONYMOUS_ENABLED" "value":"true"] map["name":"GF_AUTH_ANONYMOUS_ORG_ROLE" "value":"Admin"] map["name":"GF_SERVER_ROOT_URL" "value":"/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy/"]] "image":"k8s.gcr.io/heapster-grafana-amd64:v4.4.3" "name":"grafana" "ports":[map["containerPort":'\u0bb8' "name":"ui"]] "resources":map["limits":map["cpu":"100m" "memory":"100Mi"] "requests":map["cpu":"100m" "memory":"100Mi"]] "volumeMounts":[map["mountPath":"/var" "name":"grafana-persistent-storage"]]]] "priorityClassName":"system-cluster-critical" "tolerations":[map["effect":"NoSchedule" "key":"node-role.kubernetes.io/master"] map["key":"CriticalAddonsOnly" "operator":"Exists"]] "volumes":[map["emptyDir":map[] "name":"influxdb-persistent-storage"] map["emptyDir":map[] "name":"grafana-persistent-storage"]]]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/apps/v1/namespaces/kube-system/deployments/monitoring-influxdb-grafana-v4: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "heapster", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"ServiceAccount" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true"] "name":"heapster" "namespace":"kube-system"]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/serviceaccounts/heapster: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "heapster", Namespace: ""
Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1" "kind":"ClusterRoleBinding" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile"] "name":"heapster"] "roleRef":map["apiGroup":"rbac.authorization.k8s.io" "kind":"ClusterRole" "name":"system:heapster"] "subjects":[map["kind":"ServiceAccount" "name":"heapster" "namespace":"kube-system"]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/heapster: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "heapster-config", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "data":map["NannyConfiguration":"apiVersion: nannyconfig/v1alpha1\nkind: NannyConfiguration"] "kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"EnsureExists" "kubernetes.io/cluster-service":"true"] "name":"heapster-config" "namespace":"kube-system"]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/configmaps/heapster-config: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "eventer-config", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "data":map["NannyConfiguration":"apiVersion: nannyconfig/v1alpha1\nkind: NannyConfiguration"] "kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"EnsureExists" "kubernetes.io/cluster-service":"true"] "name":"eventer-config" "namespace":"kube-system"]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/configmaps/eventer-config: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "heapster-v1.5.2", Namespace: "kube-system"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "k8s-app":"heapster" "kubernetes.io/cluster-service":"true" "version":"v1.5.2"] "name":"heapster-v1.5.2" "namespace":"kube-system"] "spec":map["replicas":'\x01' "selector":map["matchLabels":map["k8s-app":"heapster" "version":"v1.5.2"]] "template":map["metadata":map["annotations":map["scheduler.alpha.kubernetes.io/critical-pod":""] "labels":map["k8s-app":"heapster" "version":"v1.5.2"]] "spec":map["containers":[map["command":["/heapster" "--source=kubernetes.summary_api:''" "--sink=influxdb:http://monitoring-influxdb:8086"] "image":"k8s.gcr.io/heapster-amd64:v1.5.2" "livenessProbe":map["httpGet":map["path":"/healthz" "port":'\u1f92' "scheme":"HTTP"] "initialDelaySeconds":'\u00b4' "timeoutSeconds":'\x05'] "name":"heapster"] map["command":["/eventer" "--source=kubernetes:''" "--sink=influxdb:http://monitoring-influxdb:8086"] "image":"k8s.gcr.io/heapster-amd64:v1.5.2" "name":"eventer"] map["command":["/pod_nanny" "--config-dir=/etc/config" "--cpu=80m" "--extra-cpu=0.5m" "--memory=140Mi" "--extra-memory=4Mi" "--threshold=5" "--deployment=heapster-v1.5.2" "--container=heapster" "--poll-period=300000" "--estimator=exponential"] "env":[map["name":"MY_POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"MY_POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"cdkbot/addon-resizer-amd64:1.8.1" "name":"heapster-nanny" "resources":map["limits":map["cpu":"50m" "memory":"92360Ki"] "requests":map["cpu":"50m" "memory":"92360Ki"]] "volumeMounts":[map["mountPath":"/etc/config" "name":"heapster-config-volume"]]] map["command":["/pod_nanny" "--config-dir=/etc/config" "--cpu=100m" "--extra-cpu=0m" "--memory=190Mi" "--extra-memory=500Ki" "--threshold=5" "--deployment=heapster-v1.5.2" "--container=eventer" "--poll-period=300000" "--estimator=exponential"] "env":[map["name":"MY_POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"MY_POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"cdkbot/addon-resizer-amd64:1.8.1" "name":"eventer-nanny" "resources":map["limits":map["cpu":"50m" "memory":"92360Ki"] "requests":map["cpu":"50m" "memory":"92360Ki"]] "volumeMounts":[map["mountPath":"/etc/config" "name":"eventer-config-volume"]]]] "priorityClassName":"system-cluster-critical" "serviceAccountName":"heapster" "tolerations":[map["key":"CriticalAddonsOnly" "operator":"Exists"]] "volumes":[map["configMap":map["name":"heapster-config"] "name":"heapster-config-volume"] map["configMap":map["name":"eventer-config"] "name":"eventer-config-volume"]]]]]]}
from server for: "/home/*user*/snap/microk8s/920/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/apps/v1/namespaces/kube-system/deployments/heapster-v1.5.2: dial tcp 127.0.0.1:16443: connect: connection refused
Failed to enable dashboard
I successfully accessed the Dashboard when I reinstalled the snap again and switched the dns and dashboard order.
- Didn't work:
microk8s.enable dns dashboard registry - Worked:
microk8s.enable dashboard dns registry
NumesSanguis
on 25 Sep 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
cnadeau
·
4Comments
mthaddon
·
5Comments
jareks
·
3Comments
linxuyalun
·
3Comments
toxsick
·
4Comments
Most helpful comment
Thank you for reporting this @NumesSanguis
A fix is on the way to upgrade the dashboard. For now you should be able to deploy the latest dashboard with: