Minikube: certificate apiserver-kubelet-client not signed by CA certificate ca: crypto/rsa: verification error

Created on 22 Jul 2019 · 8Comments · Source: kubernetes/minikube

I am not able to start minikube on GCP vm instance. Here is the error

root@i-ao-kubeflow-akash:~# minikube start --vm-driver=none
There is a newer version of minikube available (v1.2.0). Download it here:
https://github.com/kubernetes/minikube/releases/tag/v1.2.0

To disable this notification, run the following:
minikube config set WantUpdateNotification false
😄 minikube v1.1.1 on linux (amd64)
🔥 Creating none VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
🐳 Configuring environment for Kubernetes v1.14.3 on Docker 18.06.2-ce
❌ Unable to load cached images: loading cached images: loading image /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1: stat /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1: no such file or directory
💾 Downloading kubelet v1.14.3
💾 Downloading kubeadm v1.14.3
🚜 Pulling images ...
🚀 Launching Kubernetes ...

💣 Error starting cluster: cmd failed: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--data-minikube,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap

: running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--data-minikube,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap
output: [init] Using Kubernetes version: v1.14.3
[preflight] Running pre-flight checks
[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml already exists
[WARNING Hostname]: hostname "minikube" could not be reached
[WARNING Hostname]: hostname "minikube": lookup minikube on 169.254.169.254:53: no such host
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs/"
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using existing ca certificate authority
error execution phase certs/apiserver-kubelet-client: [certs] certificate apiserver-kubelet-client not signed by CA certificate ca: crypto/rsa: verification error
: running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--data-minikube,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap
.: exit status 1

😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:

triagneeds-information

Source

Akashdesarda

Most helpful comment

I was able to make it work on my environment by reinstalling it but making sure I removed everything related to kube as the following:

kubeadm reset
sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*
sudo apt-get autoremove
sudo rm -rf ~/.kube
sudo rm -rf ~/.minikube
sudo rm -rf /var/lib/minikube
sudo rm -rf /var/lib/kubelet
sudo rm -rf /var/lib/localkube
sudo rm -rf /data/minikube
sudo rm -rf /var/lib/kubeadm.yaml

brunomarq on 18 Sep 2019

👍10

All 8 comments

I am curious id this error happens in the newest version too ? would you mind trying with the latest verison of minikbue?

medyagh on 22 Jul 2019

same error, anyone can help on that?

9527andy on 26 Jul 2019

I was able to make it work on my environment by reinstalling it but making sure I removed everything related to kube as the following:

brunomarq on 18 Sep 2019

👍10

Hopefully it's OK if I close this - there wasn't enough information to make it actionable, and some time has already passed. If you are able to provide additional details, you may reopen it at any point by adding /reopen to your comment.

Here is additional information that may be helpful to us:

Whether the issue occurs with the latest minikube release
The exact minikube start command line used
The full output of the minikube start command, preferably with --alsologtostderr -v=3 for extra logging.

Thank you for sharing your experience!

tstromberg on 20 Sep 2019

@tstromberg I encountered this issue on the latest minikube release- 1.5.2
Answers to the bits you had requested for:

1) Yes.
2) minikube start --vm-driver=none --apiserver-ips 127.0.0.1 --apiserver-name localhost --alsologtostderr -v=3
3) Full output of the above command:

`
W1114 13:24:20.488293 14024 root.go:241] Error reading config file at /root/.minikube/config/config.json: open /root/.minikube/config/config.json: no such file or directory
I1114 13:24:20.488532 14024 notify.go:125] Checking for updates...
I1114 13:24:20.600205 14024 start.go:251] hostinfo: {"hostname":"msm-dev","uptime":1223348,"bootTime":1572514512,"procs":211,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"18.04","kernelVersion":"5.0.0-1023-azure","virtualizationSystem":"kvm","virtualizationRole":"host","hostid":"6bd5c0e5-fa1b-1a46-85c8-8b1d06c8b597"}
I1114 13:24:20.600742 14024 start.go:261] virtualization: kvm host
😄 minikube v1.5.2 on Ubuntu 18.04
I1114 13:24:20.601174 14024 start.go:547] selectDriver: flag="none", old=&{{false false https://storage.googleapis.com/minikube/iso/minikube-v1.5.1.iso 2000 2 20000 none docker [] [] [] [] 192.168.99.1/24 default qemu:///system false false [] false [] /nfsshares false false true} {v1.16.2 192.168.2.4 8443 minikube minikubeCA [] [] cluster.local docker 10.96.0.0/12 [{kubelet resolv-conf /run/systemd/resolve/resolv.conf}] true false}}
I1114 13:24:20.601655 14024 start.go:293] selected: none
I1114 13:24:20.601909 14024 profile.go:82] Saving config to /root/.minikube/profiles/minikube/config.json ...
I1114 13:24:20.602012 14024 lock.go:41] attempting to write to file "/root/.minikube/profiles/minikube/config.json.tmp666338286" with filemode -rw-------
I1114 13:24:20.602482 14024 cluster.go:101] Skipping create...Using existing machine configuration
💡 Tip: Use 'minikube start -p ' to create a new cluster, or 'minikube delete' to delete this one.
I1114 13:24:20.602753 14024 none.go:257] checking for running kubelet ...
I1114 13:24:20.602804 14024 exec_runner.go:42] (ExecRunner) Run: systemctl is-active --quiet service kubelet
I1114 13:24:20.603154 14024 cache_images.go:296] CacheImage: gcr.io/k8s-minikube/storage-provisioner:v1.8.1 -> /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1
I1114 13:24:20.603233 14024 cache_images.go:302] /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1 exists
I1114 13:24:20.603273 14024 cache_images.go:298] CacheImage: gcr.io/k8s-minikube/storage-provisioner:v1.8.1 -> /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1 completed in 126.806µs
I1114 13:24:20.603332 14024 cache_images.go:83] CacheImage gcr.io/k8s-minikube/storage-provisioner:v1.8.1 -> /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1 succeeded
I1114 13:24:20.603396 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kube-proxy:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2
I1114 13:24:20.603449 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2 exists
I1114 13:24:20.603483 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kube-proxy:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2 completed in 104.705µs
I1114 13:24:20.603525 14024 cache_images.go:83] CacheImage k8s.gcr.io/kube-proxy:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2 succeeded
I1114 13:24:20.603589 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kube-scheduler:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2
I1114 13:24:20.603716 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2 exists
I1114 13:24:20.603827 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kube-scheduler:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2 completed in 256.212µs
I1114 13:24:20.603937 14024 cache_images.go:83] CacheImage k8s.gcr.io/kube-scheduler:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2 succeeded
I1114 13:24:20.604094 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kube-controller-manager:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2
I1114 13:24:20.604224 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2 exists
I1114 13:24:20.604334 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kube-controller-manager:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2 completed in 249.112µs
I1114 13:24:20.604447 14024 cache_images.go:83] CacheImage k8s.gcr.io/kube-controller-manager:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2 succeeded
I1114 13:24:20.604568 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kube-apiserver:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2
I1114 13:24:20.604692 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2 exists
I1114 13:24:20.604803 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kube-apiserver:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2 completed in 242.012µs
I1114 13:24:20.604915 14024 cache_images.go:83] CacheImage k8s.gcr.io/kube-apiserver:v1.16.2 -> /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2 succeeded
I1114 13:24:20.605066 14024 cache_images.go:296] CacheImage: k8s.gcr.io/pause:3.1 -> /root/.minikube/cache/images/k8s.gcr.io/pause_3.1
I1114 13:24:20.605199 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/pause_3.1 exists
I1114 13:24:20.605312 14024 cache_images.go:298] CacheImage: k8s.gcr.io/pause:3.1 -> /root/.minikube/cache/images/k8s.gcr.io/pause_3.1 completed in 253.612µs
I1114 13:24:20.605425 14024 cache_images.go:83] CacheImage k8s.gcr.io/pause:3.1 -> /root/.minikube/cache/images/k8s.gcr.io/pause_3.1 succeeded
I1114 13:24:20.605539 14024 cache_images.go:296] CacheImage: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13
I1114 13:24:20.605649 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13 exists
I1114 13:24:20.605749 14024 cache_images.go:298] CacheImage: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13 completed in 217.211µs
I1114 13:24:20.605846 14024 cache_images.go:83] CacheImage k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13 succeeded
I1114 13:24:20.605954 14024 cache_images.go:296] CacheImage: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13
I1114 13:24:20.606112 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13 exists
I1114 13:24:20.606222 14024 cache_images.go:298] CacheImage: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13 completed in 274.913µs
I1114 13:24:20.606325 14024 cache_images.go:83] CacheImage k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13 succeeded
I1114 13:24:20.606429 14024 cache_images.go:296] CacheImage: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13
I1114 13:24:20.606541 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13 exists
I1114 13:24:20.606645 14024 cache_images.go:298] CacheImage: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13 completed in 222.71µs
I1114 13:24:20.606742 14024 cache_images.go:83] CacheImage k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13 -> /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13 succeeded
I1114 13:24:20.606843 14024 cache_images.go:296] CacheImage: k8s.gcr.io/etcd:3.3.15-0 -> /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0
I1114 13:24:20.606983 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0 exists
I1114 13:24:20.607120 14024 cache_images.go:298] CacheImage: k8s.gcr.io/etcd:3.3.15-0 -> /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0 completed in 281.114µs
I1114 13:24:20.607226 14024 cache_images.go:83] CacheImage k8s.gcr.io/etcd:3.3.15-0 -> /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0 succeeded
I1114 13:24:20.607336 14024 cache_images.go:296] CacheImage: k8s.gcr.io/coredns:1.6.2 -> /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2
I1114 13:24:20.607448 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2 exists
I1114 13:24:20.607555 14024 cache_images.go:298] CacheImage: k8s.gcr.io/coredns:1.6.2 -> /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2 completed in 225.811µs
I1114 13:24:20.607670 14024 cache_images.go:83] CacheImage k8s.gcr.io/coredns:1.6.2 -> /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2 succeeded
I1114 13:24:20.607796 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 -> /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1
I1114 13:24:20.607920 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1 exists
I1114 13:24:20.608064 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 -> /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1 completed in 273.713µs
I1114 13:24:20.608185 14024 cache_images.go:83] CacheImage k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 -> /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1 succeeded
I1114 13:24:20.608292 14024 cache_images.go:296] CacheImage: k8s.gcr.io/kube-addon-manager:v9.0 -> /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0
I1114 13:24:20.608398 14024 cache_images.go:302] /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0 exists
I1114 13:24:20.608492 14024 cache_images.go:298] CacheImage: k8s.gcr.io/kube-addon-manager:v9.0 -> /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0 completed in 206.71µs
I1114 13:24:20.608585 14024 cache_images.go:83] CacheImage k8s.gcr.io/kube-addon-manager:v9.0 -> /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0 succeeded
I1114 13:24:20.608679 14024 cache_images.go:90] Successfully cached all images.
I1114 13:24:20.611166 14024 cluster.go:113] Machine state: Running
🏃 Using the running none "minikube" VM ...
I1114 13:24:20.611286 14024 cluster.go:131] engine options: &{ArbitraryFlags:[] DNS:[] GraphDir: Env:[] Ipv6:false InsecureRegistry:[10.96.0.0/12] Labels:[] LogLevel: StorageDriver: SelinuxEnabled:false TLSVerify:false RegistryMirror:[] InstallURL:https://get.docker.com}
⌛ Waiting for the host to be provisioned ...
I1114 13:24:20.611353 14024 cluster.go:144] configureHost: &{BaseDriver:0xc0002fb200 CommonDriver: URL: runtime:0xc00049ad80 exec:0x276da88}
I1114 13:24:20.611403 14024 cluster.go:163] none is a local driver, skipping auth/time setup
I1114 13:24:20.611423 14024 cluster.go:146] configureHost completed within 71.403µs
I1114 13:24:20.611875 14024 exec_runner.go:42] (ExecRunner) Run: nslookup kubernetes.io
I1114 13:24:20.659858 14024 exec_runner.go:42] (ExecRunner) Run: curl -sS https://k8s.gcr.io/
I1114 13:24:20.765532 14024 profile.go:82] Saving config to /root/.minikube/profiles/minikube/config.json ...
I1114 13:24:20.765631 14024 lock.go:41] attempting to write to file "/root/.minikube/profiles/minikube/config.json.tmp473955957" with filemode -rw-------
I1114 13:24:20.765980 14024 exec_runner.go:42] (ExecRunner) Run: sudo systemctl start docker
I1114 13:24:20.783568 14024 exec_runner.go:42] (ExecRunner) Run: docker version --format '{{.Server.Version}}'
🐳 Preparing Kubernetes v1.16.2 on Docker '19.03.4' ...
I1114 13:24:20.860585 14024 settings.go:124] acquiring lock: {Name:kubeconfigUpdate Clock:{} Delay:10s Timeout:0s Cancel:}
I1114 13:24:20.860694 14024 settings.go:132] Updating kubeconfig: /root/.kube/config
I1114 13:24:20.862332 14024 lock.go:41] attempting to write to file "/root/.kube/config" with filemode -rw-------
▪ kubelet.resolv-conf=/run/systemd/resolve/resolv.conf
I1114 13:24:20.862931 14024 cache_images.go:96] LoadImages start: [k8s.gcr.io/kube-proxy:v1.16.2 k8s.gcr.io/kube-scheduler:v1.16.2 k8s.gcr.io/kube-controller-manager:v1.16.2 k8s.gcr.io/kube-apiserver:v1.16.2 k8s.gcr.io/pause:3.1 k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13 k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13 k8s.gcr.io/etcd:3.3.15-0 k8s.gcr.io/coredns:1.6.2 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kube-addon-manager:v9.0 gcr.io/k8s-minikube/storage-provisioner:v1.8.1]
I1114 13:24:20.863175 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2
I1114 13:24:20.863227 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2
I1114 13:24:20.866941 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2
I1114 13:24:20.863185 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1
I1114 13:24:20.863213 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2
I1114 13:24:20.886937 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/pause_3.1
I1114 13:24:20.970064 14024 docker.go:107] Loading image: /var/lib/minikube/images/storage-provisioner_v1.8.1
I1114 13:24:20.975090 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/storage-provisioner_v1.8.1
I1114 13:24:20.991791 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13
I1114 13:24:20.995101 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13
I1114 13:24:21.006998 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13
I1114 13:24:21.013507 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0
I1114 13:24:21.018935 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2
I1114 13:24:21.027234 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1
I1114 13:24:21.027516 14024 cache_images.go:211] Loading image from cache: /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0
I1114 13:24:21.637935 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1 from cache
I1114 13:24:21.637984 14024 docker.go:107] Loading image: /var/lib/minikube/images/pause_3.1
I1114 13:24:21.638142 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/pause_3.1
I1114 13:24:21.885272 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/pause_3.1 from cache
I1114 13:24:21.885328 14024 docker.go:107] Loading image: /var/lib/minikube/images/k8s-dns-sidecar-amd64_1.14.13
I1114 13:24:21.885412 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/k8s-dns-sidecar-amd64_1.14.13
I1114 13:24:22.050596 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13 from cache
I1114 13:24:22.050645 14024 docker.go:107] Loading image: /var/lib/minikube/images/k8s-dns-dnsmasq-nanny-amd64_1.14.13
I1114 13:24:22.050743 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/k8s-dns-dnsmasq-nanny-amd64_1.14.13
I1114 13:24:22.271928 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13 from cache
I1114 13:24:22.271979 14024 docker.go:107] Loading image: /var/lib/minikube/images/kube-scheduler_v1.16.2
I1114 13:24:22.272059 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kube-scheduler_v1.16.2
I1114 13:24:22.569098 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.16.2 from cache
I1114 13:24:22.569166 14024 docker.go:107] Loading image: /var/lib/minikube/images/kube-proxy_v1.16.2
I1114 13:24:22.569246 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kube-proxy_v1.16.2
I1114 13:24:22.822595 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.16.2 from cache
I1114 13:24:22.822645 14024 docker.go:107] Loading image: /var/lib/minikube/images/kubernetes-dashboard-amd64_v1.10.1
I1114 13:24:22.822732 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kubernetes-dashboard-amd64_v1.10.1
I1114 13:24:23.211712 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1 from cache
I1114 13:24:23.211755 14024 docker.go:107] Loading image: /var/lib/minikube/images/k8s-dns-kube-dns-amd64_1.14.13
I1114 13:24:23.211840 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/k8s-dns-kube-dns-amd64_1.14.13
I1114 13:24:23.411040 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13 from cache
I1114 13:24:23.411085 14024 docker.go:107] Loading image: /var/lib/minikube/images/coredns_1.6.2
I1114 13:24:23.411192 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/coredns_1.6.2
I1114 13:24:23.601155 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2 from cache
I1114 13:24:23.601233 14024 docker.go:107] Loading image: /var/lib/minikube/images/kube-controller-manager_v1.16.2
I1114 13:24:23.601320 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kube-controller-manager_v1.16.2
I1114 13:24:24.006354 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.16.2 from cache
I1114 13:24:24.006421 14024 docker.go:107] Loading image: /var/lib/minikube/images/kube-apiserver_v1.16.2
I1114 13:24:24.006495 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kube-apiserver_v1.16.2
I1114 13:24:24.309247 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.16.2 from cache
I1114 13:24:24.309293 14024 docker.go:107] Loading image: /var/lib/minikube/images/kube-addon-manager_v9.0
I1114 13:24:24.309376 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/kube-addon-manager_v9.0
I1114 13:24:24.583770 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0 from cache
I1114 13:24:24.583822 14024 docker.go:107] Loading image: /var/lib/minikube/images/etcd_3.3.15-0
I1114 13:24:24.583899 14024 exec_runner.go:42] (ExecRunner) Run: docker load -i /var/lib/minikube/images/etcd_3.3.15-0
I1114 13:24:25.075603 14024 cache_images.go:237] Successfully loaded image /root/.minikube/cache/images/k8s.gcr.io/etcd_3.3.15-0 from cache
I1114 13:24:25.075644 14024 cache_images.go:120] Successfully loaded all cached images.
I1114 13:24:25.075668 14024 cache_images.go:121] LoadImages end
I1114 13:24:25.075976 14024 kubeadm.go:665] kubelet v1.16.2 config:
[Unit]
Wants=docker.socket

[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.2/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=docker --fail-swap-on=false --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.2.4 --pod-manifest-path=/etc/kubernetes/manifests --resolv-conf=/run/systemd/resolve/resolv.conf

[Install]
I1114 13:24:25.076025 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "pgrep kubelet && sudo systemctl stop kubelet"
I1114 13:24:25.145949 14024 cache_binaries.go:74] Not caching binary, using https://storage.googleapis.com/kubernetes-release/release/v1.16.2/bin/linux/amd64/kubelet
I1114 13:24:25.147108 14024 cache_binaries.go:74] Not caching binary, using https://storage.googleapis.com/kubernetes-release/release/v1.16.2/bin/linux/amd64/kubeadm
I1114 13:24:25.386644 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo systemctl daemon-reload && sudo systemctl start kubelet"
I1114 13:24:25.608256 14024 certs.go:75] acquiring lock: {Name:setupCerts Clock:{} Delay:15s Timeout:0s Cancel:}
I1114 13:24:25.608424 14024 certs.go:83] Setting up /root/.minikube for IP: 192.168.2.4
I1114 13:24:25.608509 14024 crypto.go:69] Generating cert /root/.minikube/client.crt with IP's: []
I1114 13:24:25.616012 14024 crypto.go:157] Writing cert to /root/.minikube/client.crt ...
I1114 13:24:25.616345 14024 lock.go:41] attempting to write to file "/root/.minikube/client.crt" with filemode -rw-r--r--
I1114 13:24:25.620143 14024 crypto.go:165] Writing key to /root/.minikube/client.key ...
I1114 13:24:25.620173 14024 lock.go:41] attempting to write to file "/root/.minikube/client.key" with filemode -rw-------
I1114 13:24:25.620641 14024 crypto.go:69] Generating cert /root/.minikube/apiserver.crt with IP's: [127.0.0.1 192.168.2.4 10.96.0.1 10.0.0.1]
I1114 13:24:25.634691 14024 crypto.go:157] Writing cert to /root/.minikube/apiserver.crt ...
I1114 13:24:25.634806 14024 lock.go:41] attempting to write to file "/root/.minikube/apiserver.crt" with filemode -rw-r--r--
I1114 13:24:25.635128 14024 crypto.go:165] Writing key to /root/.minikube/apiserver.key ...
I1114 13:24:25.635212 14024 lock.go:41] attempting to write to file "/root/.minikube/apiserver.key" with filemode -rw-------
I1114 13:24:25.635567 14024 crypto.go:69] Generating cert /root/.minikube/proxy-client.crt with IP's: []
I1114 13:24:25.644018 14024 crypto.go:157] Writing cert to /root/.minikube/proxy-client.crt ...
I1114 13:24:25.644135 14024 lock.go:41] attempting to write to file "/root/.minikube/proxy-client.crt" with filemode -rw-r--r--
I1114 13:24:25.644509 14024 crypto.go:165] Writing key to /root/.minikube/proxy-client.key ...
I1114 13:24:25.644616 14024 lock.go:41] attempting to write to file "/root/.minikube/proxy-client.key" with filemode -rw-------
I1114 13:24:25.648735 14024 exec_runner.go:42] (ExecRunner) Run: openssl version
I1114 13:24:25.653796 14024 exec_runner.go:42] (ExecRunner) Run: sudo test -f /etc/ssl/certs/minikubeCA.pem
I1114 13:24:25.662179 14024 exec_runner.go:42] (ExecRunner) Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I1114 13:24:25.666876 14024 exec_runner.go:42] (ExecRunner) Run: sudo test -f /etc/ssl/certs/b5213941.0
🔄 Relaunching Kubernetes using kubeadm ...
I1114 13:24:25.683561 14024 kubeadm.go:436] RestartCluster start
I1114 13:24:25.683602 14024 exec_runner.go:42] (ExecRunner) Run: sudo test -d /data/minikube
I1114 13:24:25.707162 14024 exec_runner.go:74] (ExecRunner) Non-zero exit: sudo test -d /data/minikube: exit status 1 (23.540318ms)
I1114 13:24:25.707220 14024 kubeadm.go:229] /data/minikube skipping compat symlinks: command failed: sudo test -d /data/minikube
stdout:
stderr: : exit status 1
I1114 13:24:25.707252 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"
I1114 13:24:25.892125 14024 exec_runner.go:74] (ExecRunner) Non-zero exit: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml": exit status 1 (184.797379ms)
-- stdout --
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk

-- /stdout --
* stderr *
error execution phase certs/apiserver-kubelet-client: [certs] certificate apiserver-kubelet-client not signed by CA certificate ca: crypto/rsa: verification error
To see the stack trace of this error execute with --v=5 or higher

* /stderr *
I1114 13:24:25.892354 14024 kubeadm.go:439] RestartCluster took 208.781718ms
I1114 13:24:25.892436 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kube-apiserver I1114 13:24:26.003676 14024 logs.go:178] 1 containers: ["1d417ea67860"]
I1114 13:24:26.003775 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_coredns I1114 13:24:26.132024 14024 logs.go:178] 0 containers: []
W1114 13:24:26.132060 14024 logs.go:180] No container was found matching "coredns"
I1114 13:24:26.132109 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kube-scheduler I1114 13:24:26.208543 14024 logs.go:178] 1 containers: ["e754105051f3"]
I1114 13:24:26.208785 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kube-proxy I1114 13:24:26.282346 14024 logs.go:178] 0 containers: []
W1114 13:24:26.282379 14024 logs.go:180] No container was found matching "kube-proxy"
I1114 13:24:26.282438 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kube-addon-manager I1114 13:24:26.396455 14024 logs.go:178] 1 containers: ["72d45f2eb21b"]
I1114 13:24:26.396540 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kubernetes-dashboard I1114 13:24:26.482722 14024 logs.go:178] 0 containers: []
W1114 13:24:26.482802 14024 logs.go:180] No container was found matching "kubernetes-dashboard"
I1114 13:24:26.482908 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_storage-provisioner I1114 13:24:26.569399 14024 logs.go:178] 0 containers: []
W1114 13:24:26.569438 14024 logs.go:180] No container was found matching "storage-provisioner"
I1114 13:24:26.569489 14024 exec_runner.go:42] (ExecRunner) Run: docker ps -a --filter=name=k8s_kube-controller-manager I1114 13:24:26.740500 14024 logs.go:178] 1 containers: ["18dc9394512b"]
I1114 13:24:26.740556 14024 logs.go:92] Gathering logs for kube-scheduler ["e754105051f3"] ...
I1114 13:24:26.740576 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "docker logs --tail 200 I1114 13:24:26.964303 14024 logs.go:92] Gathering logs for kube-addon-manager ["72d45f2eb21b"] ...
I1114 13:24:26.964694 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "docker logs --tail 200 I1114 13:24:27.055159 14024 logs.go:92] Gathering logs for kube-controller-manager ["18dc9394512b"] ...
I1114 13:24:27.055200 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "docker logs --tail 200 I1114 13:24:27.147385 14024 logs.go:92] Gathering logs for Docker ...
I1114 13:24:27.147439 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo journalctl -u docker I1114 13:24:27.158392 14024 logs.go:92] Gathering logs for container status ...
I1114 13:24:27.158424 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo crictl ps -a || I1114 13:24:27.219211 14024 logs.go:92] Gathering logs for kubelet ...
I1114 13:24:27.219251 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo journalctl -u kubelet I1114 13:24:27.252120 14024 logs.go:92] Gathering logs for dmesg ...
I1114 13:24:27.252228 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "sudo dmesg -PH -L=never I1114 13:24:27.262684 14024 logs.go:92] Gathering logs for kube-apiserver ["1d417ea67860"] ...
I1114 13:24:27.262712 14024 exec_runner.go:42] (ExecRunner) Run: /bin/bash -c "docker logs --tail 200 W1114 13:24:27.320008 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.533214 W1114 13:24:27.320101 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.621720 W1114 13:24:27.320176 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.630150 W1114 13:24:27.320260 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.634540 W1114 13:24:27.320358 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.660503 W1114 13:24:27.320912 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.669650 W1114 13:24:27.321141 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.692147 W1114 13:24:27.321329 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.714508 W1114 13:24:27.321628 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.742054 W1114 13:24:27.321874 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.746918 W1114 13:24:27.322060 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.767423 W1114 13:24:27.322332 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.787265 W1114 13:24:27.323023 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.787759 W1114 13:24:27.323466 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:15.932364 W1114 13:24:27.323611 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.136904 W1114 13:24:27.323772 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.338360 W1114 13:24:27.323894 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.537973 W1114 13:24:27.324005 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.626668 W1114 13:24:27.324131 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.634561 W1114 13:24:27.324233 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.640473 W1114 13:24:27.324477 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.664729 W1114 13:24:27.324658 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.674219 W1114 13:24:27.324780 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.697601 W1114 13:24:27.324894 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.719702 W1114 13:24:27.325005 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.746319 W1114 13:24:27.325099 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.751986 W1114 13:24:27.325191 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.771499 W1114 13:24:27.325285 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.792574 W1114 13:24:27.325433 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.796282 W1114 13:24:27.325518 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:16.939327 W1114 13:24:27.325588 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.142307 W1114 13:24:27.325677 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.343081 W1114 13:24:27.325775 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.482390 W1114 13:24:27.325890 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.545034 W1114 13:24:27.325984 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.632503 W1114 13:24:27.326282 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.639558 W1114 13:24:27.326394 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.645021 W1114 13:24:27.326488 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.669797 W1114 13:24:27.326591 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.680427 W1114 13:24:27.326688 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.702729 W1114 13:24:27.326916 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.724803 W1114 13:24:27.327021 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.730533 W1114 13:24:27.327117 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.751365 W1114 13:24:27.327210 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.756755 W1114 13:24:27.327335 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.776105 W1114 13:24:27.327451 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.801289 W1114 13:24:27.327547 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.801963 W1114 13:24:27.327642 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:17.943478 W1114 13:24:27.327735 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.149337 W1114 13:24:27.327827 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.219531 W1114 13:24:27.327939 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.347799 W1114 13:24:27.328055 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.549439 W1114 13:24:27.328179 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.637843 W1114 13:24:27.328290 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.643842 W1114 13:24:27.328427 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.649198 W1114 13:24:27.328551 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.674088 W1114 13:24:27.328646 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.684425 W1114 13:24:27.328737 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.706828 W1114 13:24:27.328830 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.729540 W1114 13:24:27.328922 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.755872 W1114 13:24:27.329014 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.761031 W1114 13:24:27.329309 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.781251 W1114 13:24:27.330009 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.807251 W1114 13:24:27.330440 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.808270 W1114 13:24:27.330538 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:18.948296 W1114 13:24:27.330703 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.156713 W1114 13:24:27.330820 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.352391 W1114 13:24:27.330941 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.554070 W1114 13:24:27.331044 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.642451 W1114 13:24:27.331140 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.647801 W1114 13:24:27.331234 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.653393 W1114 13:24:27.331329 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.678350 W1114 13:24:27.331444 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.688707 W1114 13:24:27.331540 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.713688 W1114 13:24:27.331630 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.734853 W1114 13:24:27.331724 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.762483 W1114 13:24:27.331819 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.768207 W1114 13:24:27.331917 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.785656 W1114 13:24:27.332014 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.811969 W1114 13:24:27.332108 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.815589 W1114 13:24:27.332202 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:19.953537 W1114 13:24:27.332296 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.108315 W1114 13:24:27.332414 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.161062 W1114 13:24:27.332511 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.383564 W1114 13:24:27.332605 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.522675 W1114 13:24:27.332701 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.647625 W1114 13:24:27.332795 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.652809 W1114 13:24:27.332888 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.658385 W1114 13:24:27.332983 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.684510 W1114 13:24:27.333077 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.693649 W1114 13:24:27.333172 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.724723 W1114 13:24:27.333267 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.728650 W1114 13:24:27.333382 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.739869 W1114 13:24:27.333482 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.775634 W1114 13:24:27.333577 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.792395 W1114 13:24:27.333672 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.832548 W1114 13:24:27.333767 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.837107 W1114 13:24:27.333862 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:20.924646 W1114 13:24:27.333957 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.127400 W1114 13:24:27.334051 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.324566 W1114 13:24:27.334179 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.525320 W1114 13:24:27.334292 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.655187 W1114 13:24:27.334446 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.670170 W1114 13:24:27.334553 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.672121 W1114 13:24:27.334687 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.691704 W1114 13:24:27.334797 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.700661 W1114 13:24:27.334941 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.739741 W1114 13:24:27.335116 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.747019 W1114 13:24:27.335226 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.748858 W1114 13:24:27.340339 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.783706 W1114 13:24:27.340539 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.799025 W1114 13:24:27.340690 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.848770 W1114 13:24:27.340814 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.849725 W1114 13:24:27.341054 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:21.932598 W1114 13:24:27.341182 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.145712 W1114 13:24:27.341342 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.332270 W1114 13:24:27.341797 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.534736 W1114 13:24:27.341913 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.661963 W1114 13:24:27.342000 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.675379 W1114 13:24:27.342085 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.683092 W1114 13:24:27.342169 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.696691 W1114 13:24:27.342253 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.705671 W1114 13:24:27.342337 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.744516 W1114 13:24:27.342464 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.753992 W1114 13:24:27.342556 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.754537 W1114 13:24:27.342639 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.791196 W1114 13:24:27.342719 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.803934 W1114 13:24:27.342791 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.854568 W1114 13:24:27.342869 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.855997 W1114 13:24:27.343109 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:22.940192 W1114 13:24:27.343190 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.153666 W1114 13:24:27.343266 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.360480 W1114 13:24:27.343342 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.540495 W1114 13:24:27.343448 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.668408 W1114 13:24:27.343521 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.681490 W1114 13:24:27.343592 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.687434 W1114 13:24:27.343665 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.703321 W1114 13:24:27.343738 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.710701 W1114 13:24:27.343812 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.756086 W1114 13:24:27.343885 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.769239 W1114 13:24:27.343964 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.770964 W1114 13:24:27.344039 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.798670 W1114 13:24:27.344119 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.812076 W1114 13:24:27.344194 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.868940 W1114 13:24:27.344271 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.870939 W1114 13:24:27.344338 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.937734 W1114 13:24:27.344423 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:23.951488 W1114 13:24:27.344491 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.160750 W1114 13:24:27.344552 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.366385 W1114 13:24:27.344614 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.547186 W1114 13:24:27.344676 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.675909 W1114 13:24:27.344738 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.687142 W1114 13:24:27.344799 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.695629 W1114 13:24:27.344861 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.709613 W1114 13:24:27.344922 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.718511 W1114 13:24:27.345432 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.762385 W1114 13:24:27.345517 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.774289 W1114 13:24:27.345594 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.777193 W1114 13:24:27.345672 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.806255 W1114 13:24:27.345752 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.817020 W1114 13:24:27.345829 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.875638 W1114 13:24:27.345913 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.879838 W1114 13:24:27.345999 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:24.955390 W1114 13:24:27.346073 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.682791 W1114 13:24:27.346146 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.694340 W1114 13:24:27.346224 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.703552 W1114 13:24:27.346299 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.715148 W1114 13:24:27.346407 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.724811 W1114 13:24:27.346483 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.769843 W1114 13:24:27.346563 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.781047 W1114 13:24:27.346641 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.817241 W1114 13:24:27.346715 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.831555 W1114 13:24:27.346791 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.886394 W1114 13:24:27.346866 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:25.903072 W1114 13:24:27.346951 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.109373 W1114 13:24:27.347043 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.118783 W1114 13:24:27.347121 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.120347 W1114 13:24:27.347199 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.288684 W1114 13:24:27.347274 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.333734 W1114 13:24:27.347369 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.334681 W1114 13:24:27.347445 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.346929 W1114 13:24:27.347522 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.516854 W1114 13:24:27.347596 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.548960 W1114 13:24:27.347668 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.690546 W1114 13:24:27.347738 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.703132 W1114 13:24:27.347812 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.710489 W1114 13:24:27.347884 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.731876 W1114 13:24:27.347958 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.738408 W1114 13:24:27.348028 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.777472 W1114 13:24:27.348101 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.794021 W1114 13:24:27.348173 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.836064 W1114 13:24:27.348248 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.845777 W1114 13:24:27.348322 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.852596 W1114 13:24:27.348424 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.893670 W1114 13:24:27.348499 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.906005 W1114 13:24:27.348574 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.913233 W1114 13:24:27.348650 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:26.955721 W1114 13:24:27.348725 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:27.114630 W1114 13:24:27.348799 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:27.124726 W1114 13:24:27.348873 14024 logs.go:107] Found kube-apiserver ["1d417ea67860"] problem: I1114 13:24:27.129011 W1114 13:24:27.348909 14024 exit.go:101] Error restarting cluster: running cmd: /bin/bash -c "sudo env stdout: [certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk --format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
--format="{{.ID}}"
"e754105051f3""
"72d45f2eb21b""
"18dc9394512b""
-n 200"
sudo docker ps -a"
-n 200"
--level warn,err,crit,alert,emerg | tail -n 200"
"1d417ea67860""
1 log.go:172] http: TLS handshake error from 127.0.0.1:59872: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59874: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59876: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59878: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59880: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59882: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59884: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59886: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59888: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59890: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59892: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59894: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59896: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59900: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59902: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59904: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59910: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59912: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59914: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59916: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59918: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59920: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59922: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59924: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59926: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59928: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59930: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59932: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59934: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59936: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59940: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59942: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59944: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59946: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59950: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59952: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59954: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59956: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59958: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59960: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59962: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59964: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59966: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59968: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59970: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59974: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59972: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59978: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59980: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59984: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:59986: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60000: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60004: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60006: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60008: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60010: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60012: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60014: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60016: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60018: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60020: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60022: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60024: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60026: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60028: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60030: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60042: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60044: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60046: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60048: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60050: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60052: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60054: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60056: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60058: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60060: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60062: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60066: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60068: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60070: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60074: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60076: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60078: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60080: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60086: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60088: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60090: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60092: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60094: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60096: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60102: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60100: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60104: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60106: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60108: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60110: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60112: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60114: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60118: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60120: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60122: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60128: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60130: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60132: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60136: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60140: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60148: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60150: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60146: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60154: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60156: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60158: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60160: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60164: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60166: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60168: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60172: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60174: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60176: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60178: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60180: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60182: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60184: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60186: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60188: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60190: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60192: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60196: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60194: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60198: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60200: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60204: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60206: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60208: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60210: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60212: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60214: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60216: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60218: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60220: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60222: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60224: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60226: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60230: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60228: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60232: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60234: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60238: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60240: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60242: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60246: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60248: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60250: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60252: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60254: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60258: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60262: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60266: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60270: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60274: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60280: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60282: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60284: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60288: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60290: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60292: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60294: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60296: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60298: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60300: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60302: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60304: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60306: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60308: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60316: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60312: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60314: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60320: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60324: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60322: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60326: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60328: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60330: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60332: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60334: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60336: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60338: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60340: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60342: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60344: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60346: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60352: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60350: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60354: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60356: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60358: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60360: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60362: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60364: remote error: tls: bad certificate
1 log.go:172] http: TLS handshake error from 127.0.0.1:60366: remote error: tls: bad certificate
PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml": command failed: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"

stderr: error execution phase certs/apiserver-kubelet-client: [certs] certificate apiserver-kubelet-client not signed by CA certificate ca: crypto/rsa: verification error
To see the stack trace of this error execute with --v=5 or higher
: exit status 1

💣 Error restarting cluster: running cmd: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml": command failed: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"
stdout: [certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk

😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉 https://github.com/kubernetes/minikube/issues/new/choose
❌ Problems detected in kube-apiserver ["1d417ea67860"]:
I1114 13:24:15.533214 1 log.go:172] http: TLS handshake error from 127.0.0.1:59872: remote error: tls: bad certificate
I1114 13:24:15.621720 1 log.go:172] http: TLS handshake error from 127.0.0.1:59874: remote error: tls: bad certificate
I1114 13:24:15.630150 1 log.go:172] http: TLS handshake error from 127.0.0.1:59876: remote error: tls: bad certificate
`

vamossagar12 on 14 Nov 2019

I was able to make it work on my environment by reinstalling it but making sure I removed everything related to _kube_ as the following:

kubeadm reset
sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*
sudo apt-get autoremove
sudo rm -rf ~/.kube
sudo rm -rf ~/.minikube
sudo rm -rf /var/lib/minikube
sudo rm -rf /var/lib/kubelet
sudo rm -rf /var/lib/localkube
sudo rm -rf /data/minikube
sudo rm -rf /var/lib/kubeadm.yaml

I followed the steps outlined here and was able to start minikube. Thanks!

vamossagar12 on 3 Dec 2019

I was able to make it work on my environment by reinstalling it but making sure I removed everything related to _kube_ as the following:

kubeadm reset
sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*
sudo apt-get autoremove
sudo rm -rf ~/.kube
sudo rm -rf ~/.minikube
sudo rm -rf /var/lib/minikube
sudo rm -rf /var/lib/kubelet
sudo rm -rf /var/lib/localkube
sudo rm -rf /data/minikube
sudo rm -rf /var/lib/kubeadm.yaml

I also managed to solve it with these steps. Thank you!

guilleojeda on 11 Dec 2019

sudo rm -rf ~/.minikube
sudo rm -rf /var/lib/minikube
sudo rm -rf /var/lib/kubelet
sudo rm -rf /var/lib/localkube
sudo rm -rf /data/minikube
sudo rm -rf /var/lib/kubeadm.yaml

Worked for me as well! Thanks