Minikube: documentation: note that RBAC is disabled by default

Created on 27 Feb 2020 · 9Comments · Source: kubernetes/minikube

minikube start --extra-config=apiserver.authorization-mode=RBAC:

The full output of the command that failed:

😄 minikube v1.7.2 on Ubuntu 19.04
✨ Automatically selected the kvm2 driver. Other choices: none, docker (experimental)
🔥 Creating kvm2 VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.17.2 on Docker 19.03.5 ...
▪ apiserver.authorization-mode=RBAC
🚀 Launching Kubernetes ...

💣 Error starting cluster: init failed. output: "-- stdout --\n[init] Using Kubernetes version: v1.17.2\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Starting the kubelet\n[certs] Using certificateDir folder \"/var/lib/minikube/certs\"\n[certs] Using existing ca certificate authority\n[certs] Using existing apiserver certificate and key on disk\n[certs] Generating \"apiserver-kubelet-client\" certificate and key\n[certs] Generating \"front-proxy-ca\" certificate and key\n[certs] Generating \"front-proxy-client\" certificate and key\n[certs] Generating \"etcd/ca\" certificate and key\n[certs] Generating \"etcd/server\" certificate and key\n[certs] etcd/server serving cert is signed for DNS names [minikube localhost] and IPs [192.168.39.125 127.0.0.1 ::1]\n[certs] Generating \"etcd/peer\" certificate and key\n[certs] etcd/peer serving cert is signed for DNS names [minikube localhost] and IPs [192.168.39.125 127.0.0.1 ::1]\n[certs] Generating \"etcd/healthcheck-client\" certificate and key\n[certs] Generating \"apiserver-etcd-client\" certificate and key\n[certs] Generating \"sa\" key and public key\n[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"\n[kubeconfig] Writing \"admin.conf\" kubeconfig file\n[kubeconfig] Writing \"kubelet.conf\" kubeconfig file\n[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file\n[kubeconfig] Writing \"scheduler.conf\" kubeconfig file\n[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"\n[control-plane] Creating static Pod manifest for \"kube-apiserver\"\n[control-plane] Creating static Pod manifest for \"kube-controller-manager\"\n[control-plane] Creating static Pod manifest for \"kube-scheduler\"\n[etcd] Creating static Pod manifest for local etcd in \"/etc/kubernetes/manifests\"\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\". This can take up to 4m0s\n[apiclient] All control plane components are healthy after 15.006234 seconds\n[upload-config] Storing the configuration used in ConfigMap \"kubeadm-config\" in the \"kube-system\" Namespace\n[kubelet] Creating a ConfigMap \"kubelet-config-1.17\" in namespace kube-system with the configuration for the kubelets in the cluster\n[kubelet-check] Initial timeout of 40s passed.\n\n-- /stdout --\n* stderr * \nW0227 10:15:51.579762 2782 validation.go:28] Cannot validate kube-proxy config - no validator is available\nW0227 10:15:51.579893 2782 validation.go:28] Cannot validate kubelet config - no validator is available\n\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/\n\t[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'\nW0227 10:15:54.204421 2782 manifests.go:214] the default kube-apiserver authorization-mode is \"Node,RBAC\"; using \"RBAC\"\nW0227 10:15:54.208648 2782 manifests.go:214] the default kube-apiserver authorization-mode is \"Node,RBAC\"; using \"RBAC\"\nW0227 10:15:54.209989 2782 manifests.go:214] the default kube-apiserver authorization-mode is \"Node,RBAC\"; using \"RBAC\"\nerror execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: timed out waiting for the condition\nTo see the stack trace of this error execute with --v=5 or higher\n\n* /stderr *": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.17.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.17.2
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [minikube localhost] and IPs [192.168.39.125 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [minikube localhost] and IPs [192.168.39.125 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 15.006234 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
[kubelet-check] Initial timeout of 40s passed.

stderr:
W0227 10:15:51.579762 2782 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0227 10:15:51.579893 2782 validation.go:28] Cannot validate kubelet config - no validator is available
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
W0227 10:15:54.204421 2782 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "RBAC"
W0227 10:15:54.208648 2782 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "RBAC"
W0227 10:15:54.209989 2782 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "RBAC"
error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: timed out waiting for the condition
To see the stack trace of this error execute with --v=5 or higher

😿 minikube is exiting due to an error. If the above message is not useful, open an issue:
👉 https://github.com/kubernetes/minikube/issues/new/choose

The output of the minikube logs command:

==> Docker <==
-- Logs begin at Thu 2020-02-27 10:26:10 UTC, end at Thu 2020-02-27 10:29:23 UTC. --
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066482462Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066514240Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066538303Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066550296Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066557885Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066565813Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066573418Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066581023Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066592318Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066601046Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066713180Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.066781354Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067035930Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067056641Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067095040Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067104465Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067111593Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067118275Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067124489Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067131593Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067138567Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067214526Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067221615Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067255237Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067267802Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067275381Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067282082Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067395820Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067427230Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.067435189Z" level=info msg="containerd successfully booted in 0.227462s"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.071876063Z" level=info msg="parsed scheme: \"unix\"" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.071893521Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.071905566Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 }] }" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.071914216Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.072749528Z" level=info msg="parsed scheme: \"unix\"" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.072763072Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.072772374Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 }] }" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.072782615Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366346579Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366393173Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366406768Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_bps_device"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366418492Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_bps_device"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366429524Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_iops_device"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366440108Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_iops_device"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.366941737Z" level=info msg="Loading containers: start."
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.695907736Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.845148104Z" level=info msg="Loading containers: done."
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.945821538Z" level=info msg="Docker daemon" commit=633a0ea838 graphdriver(s)=overlay2 version=19.03.5
Feb 27 10:26:18 minikube dockerd[2012]: time="2020-02-27T10:26:18.946236163Z" level=info msg="Daemon has completed initialization"
Feb 27 10:26:19 minikube systemd[1]: Started Docker Application Container Engine.
Feb 27 10:26:19 minikube dockerd[2012]: time="2020-02-27T10:26:19.217796949Z" level=info msg="API listen on /var/run/docker.sock"
Feb 27 10:26:19 minikube dockerd[2012]: time="2020-02-27T10:26:19.219056112Z" level=info msg="API listen on [::]:2376"
Feb 27 10:27:03 minikube dockerd[2012]: time="2020-02-27T10:27:03.204143120Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/e94e67722eec7b1a58265b95fa8c3435295b05e3ce12bdc96cd50e5a2555fd09/shim.sock" debug=false pid=3442
Feb 27 10:27:03 minikube dockerd[2012]: time="2020-02-27T10:27:03.240058397Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/b085f32742c2ec86b8451dae547d390af445d17d498258ee699a7f6fd094935b/shim.sock" debug=false pid=3460
Feb 27 10:27:03 minikube dockerd[2012]: time="2020-02-27T10:27:03.541732892Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/f1214bb66b40be0568d3cd21c15ec3324bdbb58cf03a67d3a281523b7df74a06/shim.sock" debug=false pid=3541
Feb 27 10:27:03 minikube dockerd[2012]: time="2020-02-27T10:27:03.905774868Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/5272cb240913f74c0c0d3ed13aebec99f64d8be7d2039fdcf04357f2bc92d546/shim.sock" debug=false pid=3583
Feb 27 10:27:05 minikube dockerd[2012]: time="2020-02-27T10:27:05.254183218Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/b125c546a998abe6e644eec59a092058364f8d758ea9e2760334fbe40f07e65c/shim.sock" debug=false pid=3726
Feb 27 10:27:05 minikube dockerd[2012]: time="2020-02-27T10:27:05.317987786Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/24fbc81c1a7cb1a3f693cd550be6088225c8c81826df55bbc95827e38546289a/shim.sock" debug=false pid=3753
Feb 27 10:27:05 minikube dockerd[2012]: time="2020-02-27T10:27:05.799917158Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/704679936ad577bbccce50bfcb435cea4473c380536bd62d938eb733a3f49899/shim.sock" debug=false pid=3908
Feb 27 10:27:05 minikube dockerd[2012]: time="2020-02-27T10:27:05.834596826Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/02984b463c479cf3cf8d7775e5f8f718d125b6560b6f04671a22c34bc9212e74/shim.sock" debug=false pid=3927

==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
02984b463c479 41ef50a5f06a7 2 minutes ago Running kube-apiserver 0 5272cb240913f
704679936ad57 da5fd66c4068c 2 minutes ago Running kube-controller-manager 0 f1214bb66b40b
b125c546a998a f52d4c527ef2f 2 minutes ago Running kube-scheduler 0 b085f32742c2e
24fbc81c1a7cb 303ce5db0e90d 2 minutes ago Running etcd 0 e94e67722eec7

==> dmesg <==
[Feb27 10:26] You have booted with nomodeset. This means your GPU drivers are DISABLED
[ +0.000001] Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly
[ +0.000000] Unless you actually understand what nomodeset does, you should reboot without enabling it
[ +0.017564] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ +1.964122] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ +0.917035] NFSD: the nfsdcld client tracking upcall will be removed in 3.10. Please transition to using nfsdcltrack.
[ +1.208491] vboxguest: loading out-of-tree module taints kernel.
[ +0.003217] vboxguest: PCI device not found, probably running on physical hardware.
[ +5.602632] kauditd_printk_skb: 65 callbacks suppressed
[Feb27 10:28] NFSD: Unable to end grace period: -110

==> kernel <==
10:29:23 up 3 min, 0 users, load average: 0.64, 0.48, 0.20
Linux minikube 4.19.88 #1 SMP Tue Feb 4 22:25:03 PST 2020 x86_64 GNU/Linux
PRETTY_NAME="Buildroot 2019.02.8"

==> kube-apiserver I0227 10:27:06.949688 W0227 10:27:07.040142 W0227 10:27:07.047401 W0227 10:27:07.054428 W0227 10:27:07.068287 W0227 10:27:07.071113 W0227 10:27:07.087532 W0227 10:27:07.115648 W0227 10:27:07.115668 I0227 10:27:07.121251 I0227 10:27:07.121286 I0227 10:27:07.122496 I0227 10:27:07.122528 I0227 10:27:07.128333 I0227 10:27:07.128352 I0227 10:27:07.445628 I0227 10:27:07.445653 I0227 10:27:08.460954 I0227 10:27:08.461044 I0227 10:27:08.461206 I0227 10:27:08.461606 I0227 10:27:08.461864 I0227 10:27:08.461926 I0227 10:27:08.461986 I0227 10:27:08.463072 I0227 10:27:08.463110 I0227 10:27:08.463221 I0227 10:27:08.463232 I0227 10:27:08.464807 I0227 10:27:08.464872 I0227 10:27:08.464931 I0227 10:27:08.464971 I0227 10:27:08.461871 I0227 10:27:08.472006 I0227 10:27:08.472049 I0227 10:27:08.472071 I0227 10:27:08.472096 I0227 10:27:08.472111 I0227 10:27:08.472126 I0227 10:27:08.472138 I0227 10:27:08.472163 I0227 10:27:08.472184 E0227 10:27:08.531142 I0227 10:27:08.562157 I0227 10:27:08.563446 I0227 10:27:08.563464 I0227 10:27:08.565132 I0227 10:27:08.565200 I0227 10:27:09.460774 I0227 10:27:09.460826 I0227 10:27:09.461267 I0227 10:27:09.520525 I0227 10:27:09.555145 I0227 10:27:09.555183 I0227 10:27:11.648423 I0227 10:27:11.657732 I0227 10:27:12.205969 I0227 10:27:12.505210 W0227 10:27:12.889184 I0227 10:27:13.663476 [02984b463c47] <==
1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }]
1 genericapiserver.go:404] Skipping API batch/v2alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API discovery.k8s.io/v1alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API node.k8s.io/v1alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
1 genericapiserver.go:404] Skipping API apps/v1beta2 because it has no resources.
1 genericapiserver.go:404] Skipping API apps/v1beta1 because it has no resources.
1 plugins.go:158] Loaded 11 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,MutatingAdmissionWebhook,RuntimeClass.
1 plugins.go:161] Loaded 7 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,RuntimeClass,ResourceQuota.
1 client.go:361] parsed scheme: "endpoint"
1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }]
1 client.go:361] parsed scheme: "endpoint"
1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }]
1 client.go:361] parsed scheme: "endpoint"
1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }]
1 dynamic_cafile_content.go:166] Starting request-header::/var/lib/minikube/certs/front-proxy-ca.crt
1 dynamic_cafile_content.go:166] Starting client-ca-bundle::/var/lib/minikube/certs/ca.crt
1 dynamic_serving_content.go:129] Starting serving-cert::/var/lib/minikube/certs/apiserver.crt::/var/lib/minikube/certs/apiserver.key
1 secure_serving.go:178] Serving securely on [::]:8443
1 autoregister_controller.go:140] Starting autoregister controller
1 cache.go:32] Waiting for caches to sync for autoregister controller
1 tlsconfig.go:219] Starting DynamicServingCertificateController
1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
1 shared_informer.go:197] Waiting for caches to sync for cluster_authentication_trust_controller
1 available_controller.go:386] Starting AvailableConditionController
1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
1 apiservice_controller.go:94] Starting APIServiceRegistrationController
1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
1 crdregistration_controller.go:111] Starting crd-autoregister controller
1 shared_informer.go:197] Waiting for caches to sync for crd-autoregister
1 crd_finalizer.go:263] Starting CRDFinalizer
1 controller.go:81] Starting OpenAPI AggregationController
1 controller.go:85] Starting OpenAPI controller
1 customresource_discovery_controller.go:208] Starting DiscoveryController
1 naming_controller.go:288] Starting NamingConditionController
1 establishing_controller.go:73] Starting EstablishingController
1 nonstructuralschema_controller.go:191] Starting NonStructuralSchemaConditionController
1 apiapproval_controller.go:185] Starting KubernetesAPIApprovalPolicyConformantConditionController
1 dynamic_cafile_content.go:166] Starting client-ca-bundle::/var/lib/minikube/certs/ca.crt
1 dynamic_cafile_content.go:166] Starting request-header::/var/lib/minikube/certs/front-proxy-ca.crt
1 controller.go:151] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/192.168.39.108, ResourceVersion: 0, AdditionalErrorMsg:
1 cache.go:39] Caches are synced for autoregister controller
1 cache.go:39] Caches are synced for AvailableConditionController controller
1 shared_informer.go:204] Caches are synced for cluster_authentication_trust_controller
1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
1 shared_informer.go:204] Caches are synced for crd-autoregister
1 controller.go:107] OpenAPI AggregationController: Processing item
1 controller.go:130] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
1 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
1 storage_scheduling.go:133] created PriorityClass system-node-critical with value 2000001000
1 storage_scheduling.go:133] created PriorityClass system-cluster-critical with value 2000000000
1 storage_scheduling.go:142] all system priority classes are created successfully or already exist.
1 controller.go:606] quota admission added evaluator for: endpoints
1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
1 lease.go:224] Resetting endpoints for master service "kubernetes" to [192.168.39.108]
1 controller.go:606] quota admission added evaluator for: serviceaccounts

==> kube-controller-manager [704679936ad5] <==
I0227 10:27:18.066880 1 shared_informer.go:197] Waiting for caches to sync for token_cleaner
I0227 10:27:18.066892 1 shared_informer.go:204] Caches are synced for token_cleaner
E0227 10:27:18.316169 1 core.go:91] Failed to start service controller: WARNING: no cloud provider provided, services of type LoadBalancer will fail
W0227 10:27:18.316236 1 controllermanager.go:525] Skipping "service"
I0227 10:27:18.464374 1 node_lifecycle_controller.go:77] Sending events to api server
E0227 10:27:18.464421 1 core.go:232] failed to start cloud node lifecycle controller: no cloud provider provided
W0227 10:27:18.464444 1 controllermanager.go:525] Skipping "cloud-node-lifecycle"
I0227 10:27:18.716590 1 controllermanager.go:533] Started "persistentvolume-binder"
I0227 10:27:18.716638 1 pv_controller_base.go:294] Starting persistent volume controller
I0227 10:27:18.716905 1 shared_informer.go:197] Waiting for caches to sync for persistent volume
I0227 10:27:18.966336 1 controllermanager.go:533] Started "replicationcontroller"
I0227 10:27:18.966685 1 replica_set.go:180] Starting replicationcontroller controller
I0227 10:27:18.966847 1 shared_informer.go:197] Waiting for caches to sync for ReplicationController
I0227 10:27:19.216498 1 controllermanager.go:533] Started "podgc"
I0227 10:27:19.216675 1 gc_controller.go:88] Starting GC controller
I0227 10:27:19.216754 1 shared_informer.go:197] Waiting for caches to sync for GC
I0227 10:27:19.465911 1 controllermanager.go:533] Started "daemonset"
I0227 10:27:19.466256 1 daemon_controller.go:255] Starting daemon sets controller
I0227 10:27:19.466509 1 shared_informer.go:197] Waiting for caches to sync for daemon sets
I0227 10:27:19.717175 1 controllermanager.go:533] Started "pvc-protection"
I0227 10:27:19.717245 1 pvc_protection_controller.go:100] Starting PVC protection controller
I0227 10:27:19.717255 1 shared_informer.go:197] Waiting for caches to sync for PVC protection
I0227 10:27:20.416923 1 controllermanager.go:533] Started "horizontalpodautoscaling"
I0227 10:27:20.417756 1 horizontal.go:156] Starting HPA controller
I0227 10:27:20.417777 1 shared_informer.go:197] Waiting for caches to sync for HPA
I0227 10:27:20.420881 1 shared_informer.go:197] Waiting for caches to sync for resource quota
I0227 10:27:20.446243 1 shared_informer.go:204] Caches are synced for ReplicaSet
I0227 10:27:20.447104 1 shared_informer.go:197] Waiting for caches to sync for garbage collector
I0227 10:27:20.458949 1 shared_informer.go:204] Caches are synced for namespace
I0227 10:27:20.461147 1 shared_informer.go:204] Caches are synced for stateful set
I0227 10:27:20.466805 1 shared_informer.go:204] Caches are synced for daemon sets
I0227 10:27:20.466968 1 shared_informer.go:204] Caches are synced for ReplicationController
I0227 10:27:20.467058 1 shared_informer.go:204] Caches are synced for taint
I0227 10:27:20.467170 1 taint_manager.go:186] Starting NoExecuteTaintManager
I0227 10:27:20.468128 1 shared_informer.go:204] Caches are synced for bootstrap_signer
I0227 10:27:20.474970 1 shared_informer.go:204] Caches are synced for ClusterRoleAggregator
I0227 10:27:20.476942 1 shared_informer.go:204] Caches are synced for certificate-csrapproving
I0227 10:27:20.486030 1 shared_informer.go:204] Caches are synced for disruption
I0227 10:27:20.486160 1 disruption.go:338] Sending events to api server.
I0227 10:27:20.507734 1 shared_informer.go:204] Caches are synced for TTL
I0227 10:27:20.509154 1 shared_informer.go:204] Caches are synced for certificate-csrsigning
I0227 10:27:20.510531 1 shared_informer.go:204] Caches are synced for service account
I0227 10:27:20.517189 1 shared_informer.go:204] Caches are synced for GC
I0227 10:27:20.517749 1 shared_informer.go:204] Caches are synced for PVC protection
I0227 10:27:20.517984 1 shared_informer.go:204] Caches are synced for HPA
I0227 10:27:20.523323 1 shared_informer.go:204] Caches are synced for deployment
I0227 10:27:20.525585 1 shared_informer.go:204] Caches are synced for job
E0227 10:27:20.602574 1 clusterroleaggregation_controller.go:180] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
E0227 10:27:20.602584 1 clusterroleaggregation_controller.go:180] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
I0227 10:27:20.766983 1 shared_informer.go:204] Caches are synced for PV protection
I0227 10:27:20.817262 1 shared_informer.go:204] Caches are synced for persistent volume
I0227 10:27:20.819014 1 shared_informer.go:204] Caches are synced for attach detach
I0227 10:27:20.819782 1 shared_informer.go:204] Caches are synced for expand
E0227 10:27:20.867400 1 clusterroleaggregation_controller.go:180] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
I0227 10:27:20.883748 1 shared_informer.go:204] Caches are synced for endpoint
I0227 10:27:21.093781 1 shared_informer.go:204] Caches are synced for garbage collector
I0227 10:27:21.093811 1 garbagecollector.go:138] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0227 10:27:21.120539 1 shared_informer.go:204] Caches are synced for resource quota
I0227 10:27:21.121118 1 shared_informer.go:204] Caches are synced for resource quota
I0227 10:27:21.147999 1 shared_informer.go:204] Caches are synced for garbage collector

==> kube-scheduler [b125c546a998] <==
W0227 10:27:06.119117 1 authentication.go:298] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
W0227 10:27:06.141104 1 authorization.go:47] Authorization is disabled
W0227 10:27:06.141218 1 authentication.go:92] Authentication is disabled
I0227 10:27:06.141400 1 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251
I0227 10:27:06.142487 1 secure_serving.go:178] Serving securely on 127.0.0.1:10259
I0227 10:27:06.143318 1 configmap_cafile_content.go:205] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0227 10:27:06.143438 1 shared_informer.go:197] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0227 10:27:06.143560 1 tlsconfig.go:219] Starting DynamicServingCertificateController
E0227 10:27:06.146906 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StatefulSet: Get https://localhost:8443/apis/apps/v1/statefulsets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.147154 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.147543 1 reflector.go:153] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:209: Failed to list *v1.ConfigMap: Get https://localhost:8443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dextension-apiserver-authentication&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.148258 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: Get https://localhost:8443/api/v1/persistentvolumeclaims?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.148518 1 reflector.go:153] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:246: Failed to list *v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=status.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.148917 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: Get https://localhost:8443/api/v1/persistentvolumes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.149189 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: Get https://localhost:8443/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.149476 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicationController: Get https://localhost:8443/api/v1/replicationcontrollers?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.151151 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: Get https://localhost:8443/apis/storage.k8s.io/v1/csinodes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.151278 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: Get https://localhost:8443/api/v1/nodes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.151409 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: Get https://localhost:8443/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:06.151468 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicaSet: Get https://localhost:8443/apis/apps/v1/replicasets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: connect: connection refused
E0227 10:27:08.524308 1 reflector.go:153] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:209: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0227 10:27:08.526159 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0227 10:27:08.531956 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0227 10:27:08.534161 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0227 10:27:08.536269 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0227 10:27:08.536415 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0227 10:27:08.536533 1 reflector.go:153] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:246: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0227 10:27:08.536653 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0227 10:27:08.536759 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:08.536873 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0227 10:27:08.536961 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:08.537094 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0227 10:27:09.526786 1 reflector.go:153] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:209: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0227 10:27:09.532005 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0227 10:27:09.533787 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0227 10:27:09.538173 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0227 10:27:09.539822 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0227 10:27:09.540040 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0227 10:27:09.541545 1 reflector.go:153] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:246: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0227 10:27:09.542744 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0227 10:27:09.544193 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:09.545486 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0227 10:27:09.546442 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:09.547530 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0227 10:27:10.528040 1 reflector.go:153] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:209: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0227 10:27:10.532823 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0227 10:27:10.534549 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0227 10:27:10.538824 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0227 10:27:10.540571 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0227 10:27:10.542565 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0227 10:27:10.543151 1 reflector.go:153] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:246: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0227 10:27:10.544619 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0227 10:27:10.546529 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:10.548123 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0227 10:27:10.549299 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0227 10:27:10.550520 1 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0227 10:27:11.530032 1 reflector.go:153] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:209: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0227 10:27:11.643063 1 leaderelection.go:242] attempting to acquire leader lease kube-system/kube-scheduler...
I0227 10:27:11.663558 1 leaderelection.go:252] successfully acquired lease kube-system/kube-scheduler
I0227 10:27:12.546140 1 shared_informer.go:204] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

==> kubelet <==
-- Logs begin at Thu 2020-02-27 10:26:10 UTC, end at Thu 2020-02-27 10:29:23 UTC. --
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.616727 5691 kubelet_node_status.go:92] Unable to register node "minikube" with API server: nodes is forbidden: User "system:node:minikube" cannot create resource "nodes" in API group "" at the cluster scope
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.643422 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.743783 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.816260 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:node:minikube" cannot list resource "pods" in API group "" at the cluster scope
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.844293 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:19 minikube kubelet[5691]: E0227 10:29:19.944780 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.016442 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:node:minikube" cannot list resource "services" in API group "" at the cluster scope
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.045109 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.145485 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.216943 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "minikube" is forbidden: User "system:node:minikube" cannot list resource "nodes" in API group "" at the cluster scope
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.245926 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.346274 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.416974 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:node:minikube" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.446821 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.547019 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.616213 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:node:minikube" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.647307 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.747615 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.818498 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:node:minikube" cannot list resource "pods" in API group "" at the cluster scope
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.847789 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:20 minikube kubelet[5691]: E0227 10:29:20.948292 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.018944 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:node:minikube" cannot list resource "services" in API group "" at the cluster scope
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.048524 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.148787 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.219363 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "minikube" is forbidden: User "system:node:minikube" cannot list resource "nodes" in API group "" at the cluster scope
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.249323 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.349820 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.418811 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:node:minikube" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.450422 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.550894 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.619355 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:node:minikube" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.651700 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.752205 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.820941 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:node:minikube" cannot list resource "pods" in API group "" at the cluster scope
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.852864 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:21 minikube kubelet[5691]: E0227 10:29:21.953375 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.021028 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:node:minikube" cannot list resource "services" in API group "" at the cluster scope
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.053868 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.154228 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.222143 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "minikube" is forbidden: User "system:node:minikube" cannot list resource "nodes" in API group "" at the cluster scope
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.254782 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.355385 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.421371 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:node:minikube" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.455905 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.458012 5691 controller.go:135] failed to ensure node lease exists, will retry in 7s, error: leases.coordination.k8s.io "minikube" is forbidden: User "system:node:minikube" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.556339 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.621438 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:node:minikube" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.657615 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.758011 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.822527 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:node:minikube" cannot list resource "pods" in API group "" at the cluster scope
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.858595 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:22 minikube kubelet[5691]: E0227 10:29:22.959143 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.023028 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:node:minikube" cannot list resource "services" in API group "" at the cluster scope
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.059619 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.159976 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.224195 5691 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "minikube" is forbidden: User "system:node:minikube" cannot list resource "nodes" in API group "" at the cluster scope
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.260130 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.360288 5691 kubelet.go:2263] node "minikube" not found
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.422679 5691 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:node:minikube" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Feb 27 10:29:23 minikube kubelet[5691]: E0227 10:29:23.460431 5691 kubelet.go:2263] node "minikube" not found

The operating system version: Ubuntu 19.04

good first issue help wanted kindocumentation kinsupport

Source

lesovsky

All 9 comments

@lesovsky we recently merged a PR to improve the rbac, I believe it is not in 1.7.3 release yet bu tI am curious,

if this binary from the master would fix your issue ?
do you mind try this binary ?
https://github.com/kubernetes/minikube/suites/479659808/artifacts/2201497

medyagh on 28 Feb 2020

also in minikube by default we create an RBAC I wonder if your extra option is needed ? does your usecase work without specifying the rbac?

https://github.com/kubernetes/minikube/blob/master/pkg/minikube/bootstrapper/kubeadm/kubeadm.go#L507

medyagh on 28 Feb 2020

https://github.com/kubernetes/minikube/suites/479659808/artifacts/2201497

This binary fails to start too.

also in minikube by default we create an RBAC I wonder if your extra option is needed ?

I supposed that RBAC is disabled by default in minikube, because in the official documentation I saw a note about how to enable RBAC explicitly:

To set the AuthorizationMode on the apiserver to RBAC, you can use: --extra-config=apiserver.authorization-mode=RBAC

does your usecase work without specifying the rbac?

Now I checked RBAC with kubectl api-versions command, make sure RBAC is enabled by default and my tasks related to RBAC work as expected, so I don't need to use --extra-config=apiserver.authorization-mode=RBAC option.

I think that will be nice to improve official minikube documentation - remove a mention about apiserver.authorization-mode=RBAC and note that RBAC is enabled by default.

If case with failing minikube with "apiserver.authorization-mode=RBAC" don't worrying developers, this issue might be closed )))

p.s. 1.7.1 doesn't have such issue.

lesovsky on 29 Feb 2020

Hey @lesovsky thanks for looking into this. If you, or anyone else, would be interested in opening a PR to improve the documentation around RBAC, I'd be happy to review it!

priyawadhwa on 4 Mar 2020

I'm a bit confused by this: to the best of my knowledge, minikube does not disable RBAC.

Perhaps the upstream docs are wrong?

tstromberg on 5 Mar 2020

Just started a fresh cluster and confirmed RBAC is enabled by default.

$ minikube version
minikube version: v1.9.0
commit: 48fefd43444d2f8852f527c78f0141b377b1e42a

$ kubectl api-resources | grep rbac
clusterrolebindings   rbac.authorization.k8s.io    false    ClusterRoleBinding
clusterroles          rbac.authorization.k8s.io    false    ClusterRole
rolebindings          rbac.authorization.k8s.io    true     RoleBinding
roles                 rbac.authorization.k8s.io    true     Role

I can submit the PR upstream to kubernetes/website, @priyawadhwa.