Microk8s: Unable to enable dashboard

[batchu]

I installed microk8s through snap on ubuntu and I get the following error when enabling the dashboard.

user@ubuntu:~$ sudo snap install microk8s --classic
microk8s v1.14.2 from Canonical✓ installed
user@ubuntu:~$ watch microk8s.kubectl get all --all-namespaces
user@ubuntu:~$ microk8s.status
microk8s is running
addons:
jaeger: disabled
fluentd: disabled
gpu: disabled
storage: disabled
registry: disabled
rbac: disabled
ingress: disabled
dns: disabled
metrics-server: disabled
linkerd: disabled
prometheus: disabled
istio: disabled
dashboard: disabled
user@ubuntu:~$ microk8s.status --wait-ready
microk8s is running
addons:
jaeger: disabled
fluentd: disabled
gpu: disabled
storage: disabled
registry: disabled
rbac: disabled
ingress: disabled
dns: disabled
metrics-server: disabled
linkerd: disabled
prometheus: disabled
istio: disabled
dashboard: disabled
user@ubuntu:~$ microk8s.kubectl get nodes\
>
NAME    STATUS   ROLES    AGE   VERSION
mindy   Ready    <none>   2m    v1.14.2
user@ubuntu:~$ microk8s.kubectl get nodes
NAME    STATUS   ROLES    AGE    VERSION
mindy   Ready    <none>   2m3s   v1.14.2
user@ubuntu:~$ microk8s.kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   2m8s
user@ubuntu:~$ snap alias microk8s.kubectl kubectl
error: access denied (try with sudo)
user@ubuntu:~$ sudo snap alias microk8s.kubectl kubectl
Added:
  - microk8s.kubectl as kubectl
user@ubuntu:~$ microk8s.enable dns dashboard
Enabling DNS
Applying manifest
service/kube-dns created
serviceaccount/kube-dns created
configmap/kube-dns created
deployment.extensions/kube-dns created
Restarting kubelet
DNS is enabled
Enabling dashboard
error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "kubernetes-dashboard-certs", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Secret" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-certs" "namespace":"kube-system"] "type":"Opaque"]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/secrets/kubernetes-dashboard-certs: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"ServiceAccount" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/serviceaccounts/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "apps/v1beta2, Resource=deployments", GroupVersionKind: "apps/v1beta2, Kind=Deployment"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"apps/v1beta2" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "spec":map["replicas":'\x01' "revisionHistoryLimit":'\n' "selector":map["matchLabels":map["k8s-app":"kubernetes-dashboard"]] "template":map["metadata":map["labels":map["k8s-app":"kubernetes-dashboard"]] "spec":map["containers":[map["args":["--auto-generate-certificates"] "image":"k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3" "livenessProbe":map["httpGet":map["path":"/" "port":'\u20fb' "scheme":"HTTPS"] "initialDelaySeconds":'\x1e' "timeoutSeconds":'\x1e'] "name":"kubernetes-dashboard" "ports":[map["containerPort":'\u20fb' "protocol":"TCP"]] "volumeMounts":[map["mountPath":"/certs" "name":"kubernetes-dashboard-certs"] map["mountPath":"/tmp" "name":"tmp-volume"]]]] "serviceAccountName":"kubernetes-dashboard" "tolerations":[map["effect":"NoSchedule" "key":"node-role.kubernetes.io/master"]] "volumes":[map["name":"kubernetes-dashboard-certs" "secret":map["secretName":"kubernetes-dashboard-certs"]] map["emptyDir":map[] "name":"tmp-volume"]]]]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/apps/v1beta2/namespaces/kube-system/deployments/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "kubernetes-dashboard", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard" "namespace":"kube-system"] "spec":map["ports":[map["port":'\u01bb' "targetPort":'\u20fb']] "selector":map["k8s-app":"kubernetes-dashboard"]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/kubernetes-dashboard: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "monitoring-grafana", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"Grafana"] "name":"monitoring-grafana" "namespace":"kube-system"] "spec":map["ports":[map["port":'P' "protocol":"TCP" "targetPort":"ui"]] "selector":map["k8s-app":"influxGrafana"]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-grafana: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "monitoring-influxdb", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"InfluxDB"] "name":"monitoring-influxdb" "namespace":"kube-system"] "spec":map["ports":[map["name":"http" "port":'\u1f93' "targetPort":'\u1f93'] map["name":"api" "port":'\u1f96' "targetPort":'\u1f96']] "selector":map["k8s-app":"influxGrafana"]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/monitoring-influxdb: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "heapster", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"Service" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true" "kubernetes.io/name":"Heapster"] "name":"heapster" "namespace":"kube-system"] "spec":map["ports":[map["port":'P' "targetPort":'\u1f92']] "selector":map["k8s-app":"heapster"]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/services/heapster: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "extensions/v1beta1, Resource=deployments", GroupVersionKind: "extensions/v1beta1, Kind=Deployment"
Name: "monitoring-influxdb-grafana-v4", Namespace: "kube-system"
Object: &{map["apiVersion":"extensions/v1beta1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "k8s-app":"influxGrafana" "kubernetes.io/cluster-service":"true" "version":"v4"] "name":"monitoring-influxdb-grafana-v4" "namespace":"kube-system"] "spec":map["replicas":'\x01' "selector":map["matchLabels":map["k8s-app":"influxGrafana" "version":"v4"]] "template":map["metadata":map["annotations":map["scheduler.alpha.kubernetes.io/critical-pod":""] "labels":map["k8s-app":"influxGrafana" "version":"v4"]] "spec":map["containers":[map["image":"k8s.gcr.io/heapster-influxdb-amd64:v1.3.3" "name":"influxdb" "ports":[map["containerPort":'\u1f93' "name":"http"] map["containerPort":'\u1f96' "name":"api"]] "resources":map["limits":map["cpu":"100m" "memory":"500Mi"] "requests":map["cpu":"100m" "memory":"500Mi"]] "volumeMounts":[map["mountPath":"/data" "name":"influxdb-persistent-storage"]]] map["env":[map["name":"INFLUXDB_SERVICE_URL" "value":"http://monitoring-influxdb:8086"] map["name":"GF_AUTH_BASIC_ENABLED" "value":"false"] map["name":"GF_AUTH_ANONYMOUS_ENABLED" "value":"true"] map["name":"GF_AUTH_ANONYMOUS_ORG_ROLE" "value":"Admin"] map["name":"GF_SERVER_ROOT_URL" "value":"/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy/"]] "image":"k8s.gcr.io/heapster-grafana-amd64:v4.4.3" "name":"grafana" "ports":[map["containerPort":'\u0bb8' "name":"ui"]] "resources":map["limits":map["cpu":"100m" "memory":"100Mi"] "requests":map["cpu":"100m" "memory":"100Mi"]] "volumeMounts":[map["mountPath":"/var" "name":"grafana-persistent-storage"]]]] "priorityClassName":"system-cluster-critical" "tolerations":[map["effect":"NoSchedule" "key":"node-role.kubernetes.io/master"] map["key":"CriticalAddonsOnly" "operator":"Exists"]] "volumes":[map["emptyDir":map[] "name":"influxdb-persistent-storage"] map["emptyDir":map[] "name":"grafana-persistent-storage"]]]]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/extensions/v1beta1/namespaces/kube-system/deployments/monitoring-influxdb-grafana-v4: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "heapster", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "kind":"ServiceAccount" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "kubernetes.io/cluster-service":"true"] "name":"heapster" "namespace":"kube-system"]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/serviceaccounts/heapster: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "heapster-config", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "data":map["NannyConfiguration":"apiVersion: nannyconfig/v1alpha1\nkind: NannyConfiguration"] "kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"EnsureExists" "kubernetes.io/cluster-service":"true"] "name":"heapster-config" "namespace":"kube-system"]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/configmaps/heapster-config: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "eventer-config", Namespace: "kube-system"
Object: &{map["apiVersion":"v1" "data":map["NannyConfiguration":"apiVersion: nannyconfig/v1alpha1\nkind: NannyConfiguration"] "kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"EnsureExists" "kubernetes.io/cluster-service":"true"] "name":"eventer-config" "namespace":"kube-system"]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/api/v1/namespaces/kube-system/configmaps/eventer-config: dial tcp 127.0.0.1:16443: connect: connection refused
error when retrieving current configuration of:
Resource: "extensions/v1beta1, Resource=deployments", GroupVersionKind: "extensions/v1beta1, Kind=Deployment"
Name: "heapster-v1.5.2", Namespace: "kube-system"
Object: &{map["apiVersion":"extensions/v1beta1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["addonmanager.kubernetes.io/mode":"Reconcile" "k8s-app":"heapster" "kubernetes.io/cluster-service":"true" "version":"v1.5.2"] "name":"heapster-v1.5.2" "namespace":"kube-system"] "spec":map["replicas":'\x01' "selector":map["matchLabels":map["k8s-app":"heapster" "version":"v1.5.2"]] "template":map["metadata":map["annotations":map["scheduler.alpha.kubernetes.io/critical-pod":""] "labels":map["k8s-app":"heapster" "version":"v1.5.2"]] "spec":map["containers":[map["command":["/heapster" "--source=kubernetes.summary_api:''" "--sink=influxdb:http://monitoring-influxdb:8086"] "image":"k8s.gcr.io/heapster-amd64:v1.5.2" "livenessProbe":map["httpGet":map["path":"/healthz" "port":'\u1f92' "scheme":"HTTP"] "initialDelaySeconds":'\u00b4' "timeoutSeconds":'\x05'] "name":"heapster"] map["command":["/eventer" "--source=kubernetes:''" "--sink=influxdb:http://monitoring-influxdb:8086"] "image":"k8s.gcr.io/heapster-amd64:v1.5.2" "name":"eventer"] map["command":["/pod_nanny" "--config-dir=/etc/config" "--cpu=80m" "--extra-cpu=0.5m" "--memory=140Mi" "--extra-memory=4Mi" "--threshold=5" "--deployment=heapster-v1.5.2" "--container=heapster" "--poll-period=300000" "--estimator=exponential"] "env":[map["name":"MY_POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"MY_POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"cdkbot/addon-resizer-amd64:1.8.1" "name":"heapster-nanny" "resources":map["limits":map["cpu":"50m" "memory":"92360Ki"] "requests":map["cpu":"50m" "memory":"92360Ki"]] "volumeMounts":[map["mountPath":"/etc/config" "name":"heapster-config-volume"]]] map["command":["/pod_nanny" "--config-dir=/etc/config" "--cpu=100m" "--extra-cpu=0m" "--memory=190Mi" "--extra-memory=500Ki" "--threshold=5" "--deployment=heapster-v1.5.2" "--container=eventer" "--poll-period=300000" "--estimator=exponential"] "env":[map["name":"MY_POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"MY_POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"cdkbot/addon-resizer-amd64:1.8.1" "name":"eventer-nanny" "resources":map["limits":map["cpu":"50m" "memory":"92360Ki"] "requests":map["cpu":"50m" "memory":"92360Ki"]] "volumeMounts":[map["mountPath":"/etc/config" "name":"eventer-config-volume"]]]] "priorityClassName":"system-cluster-critical" "serviceAccountName":"heapster" "tolerations":[map["key":"CriticalAddonsOnly" "operator":"Exists"]] "volumes":[map["configMap":map["name":"heapster-config"] "name":"heapster-config-volume"] map["configMap":map["name":"eventer-config"] "name":"eventer-config-volume"]]]]]]}
from server for: "/home/uhsarp/snap/microk8s/608/tmp/temp.dashboard.yaml": Get https://127.0.0.1:16443/apis/extensions/v1beta1/namespaces/kube-system/deployments/heapster-v1.5.2: dial tcp 127.0.0.1:16443: connect: connection refused
Failed to enable dashboard
user@ubuntu:~$

Here's the system info

user@ubuntu:~$ sudo lshw -short
H/W path                   Device        Class       Description
================================================================
                                         system      MS-7786 (To be filled by O.E.M.)
/0                                       bus         A55M-P33 (MS-7786)
/0/0                                     memory      64KiB BIOS
/0/23                                    memory      16GiB System Memory
/0/23/0                                  memory      8GiB DIMM DDR3 1600 MHz (0.6 ns)
/0/23/1                                  memory      8GiB DIMM DDR3 1600 MHz (0.6 ns)
/0/2d                                    memory      256KiB L1 cache
/0/2e                                    memory      1MiB L2 cache
/0/30                                    processor   AMD A4-3400 APU with Radeon(tm) HD Graphics
/0/100                                   bridge      Family 12h Processor Root Complex
/0/100/1                                 display     Sumo [Radeon HD 6410D]
/0/100/1.1                               multimedia  BeaverCreek HDMI Audio [Radeon HD 6500D and 6400G-6600G series]
/0/100/4                                 bridge      Family 12h Processor Root Port
/0/100/4/0                 enp1s0        network     RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
/0/100/11                                storage     FCH SATA Controller [AHCI mode]
/0/100/12                                bus         FCH USB OHCI Controller
/0/100/12/1                usb4          bus         OHCI PCI host controller
/0/100/12.2                              bus         FCH USB EHCI Controller
/0/100/12.2/1              usb1          bus         EHCI Host Controller
/0/100/13                                bus         FCH USB OHCI Controller
/0/100/13/1                usb5          bus         OHCI PCI host controller
/0/100/13.2                              bus         FCH USB EHCI Controller
/0/100/13.2/1              usb2          bus         EHCI Host Controller
/0/100/13.2/1/4            scsi6         storage     USB 2.0 FD
/0/100/13.2/1/4/0.0.0      /dev/sdb      disk        32GB USB 2.0 FD
/0/100/13.2/1/4/0.0.0/0    /dev/sdb      disk        32GB
/0/100/13.2/1/4/0.0.0/0/1  /dev/sdb1     volume      29GiB Windows FAT volume
/0/100/13.2/1/5                          bus         USB2.0 Hub
/0/100/13.2/1/5/1                        input       USB Optical Mouse
/0/100/13.2/1/5/2                        input       Natural
/0/100/14                                bus         FCH SMBus Controller
/0/100/14.2                              multimedia  FCH Azalia Controller
/0/100/14.3                              bridge      FCH LPC Bridge
/0/100/14.4                              bridge      FCH PCI Bridge
/0/100/14.5                              bus         FCH USB OHCI Controller
/0/100/14.5/1              usb6          bus         OHCI PCI host controller
/0/100/14.7                              generic     FCH SD Flash Controller
/0/100/16                                bus         FCH USB OHCI Controller
/0/100/16/1                usb7          bus         OHCI PCI host controller
/0/100/16.2                              bus         FCH USB EHCI Controller
/0/100/16.2/1              usb3          bus         EHCI Host Controller
/0/101                                   bridge      Family 12h/14h Processor Function 0
/0/102                                   bridge      Family 12h/14h Processor Function 1
/0/103                                   bridge      Family 12h/14h Processor Function 2
/0/104                                   bridge      Family 12h/14h Processor Function 3
/0/105                                   bridge      Family 12h/14h Processor Function 4
/0/106                                   bridge      Family 12h/14h Processor Function 6
/0/107                                   bridge      Family 12h/14h Processor Function 5
/0/108                                   bridge      Family 12h/14h Processor Function 7
/0/1                       scsi5         storage
/0/1/0.0.0                 /dev/sda      disk        3TB ST3000DM001-9YN1
/0/1/0.0.0/1                             volume      511MiB Windows FAT volume
/0/1/0.0.0/2               /dev/sda2     volume      1GiB EXT4 volume
/0/1/0.0.0/3               /dev/sda3     volume      2793GiB EFI partition
/1                         vethef30159f  network     Ethernet interface
/2                         cbr0          network     Ethernet interface
user@ubuntu:~$

inspection-report-20190620_021428.tar.gz

[ktsakalozos]

@batchu if you enable first dns and then enable the dashboard do you see this? I am thinking that since dns needs to restart some daemons it may be causing some service disruption causing the dashboard to fail.

[ashishraman]

I am getting same error on CircleCI intermittently @ktsakalozos

[cyrilthank]

i executed sudo microk8s.enable dns storage dashboard
the 2nd time and it went through

[sumitkp18]

add the user multipass to the 'microk8s' group:

sudo usermod -a -G microk8s multipass

The new group will be available on the user's next login.

and then setup microk8s again:
sudo kubernetes-tools/setup-microk8s.sh

you will be able to acces the dashboard now

[ktsakalozos]

A fix for this should be on edge (sudo snap install microk8s --classic --edge) and should reach the stable channel along with v1.16.1.

[antevens]

For myself the source of this issue/error turned out to be that the interface name changed so rather than doing:
sudo ufw allow in on cbr0 && sudo ufw allow out on cbr0
Now its:
sudo ufw allow in on cni0 && sudo ufw allow out on cni0

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.