Minikube: Using `--bootstrapper=kubeadm` with `--vm-driver=none` runs into timeout

[webwurst]

Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug report

Environment:

• Minikube version (use minikube version): v0.24.1
• OS (e.g. from /etc/os-release): Ubuntu 17.10 (Artful Aardvark)
• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): none

What happened:

$ sudo CHANGE_MINIKUBE_NONE_USER=true minikube --bootstrapper kubeadm start --vm-driver=none

Starting local Kubernetes v1.8.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubeadm v1.8.0
Downloading kubelet v1.8.0
Finished Downloading kubelet v1.8.0
Finished Downloading kubeadm v1.8.0
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E1221 18:44:19.832251   11426 start.go:270] Error restarting cluster:  restarting kube-proxy: waiting for kube-proxy to be up for configmap update: timed out waiting for the condition

$ sudo docker logs --tail 3 k8s_kube-apiserver_kube-apiserver-x1_kube-system_afea2827f3bfa23dc32fedabc745119b_0
I1221 18:27:31.755848       1 logs.go:41] http: TLS handshake error from 192.168.11.23:34054: remote error: tls: bad certificate
I1221 18:27:32.047523       1 logs.go:41] http: TLS handshake error from 192.168.11.23:34058: remote error: tls: bad certificate
I1221 18:27:32.110097       1 logs.go:41] http: TLS handshake error from 192.168.11.23:34060: remote error: tls: bad certificate

$ sudo docker ps

CONTAINER ID        IMAGE                                                    COMMAND                  CREATED             STATUS              PORTS               NAMES
eee1c7da772d        gcr.io/google_containers/kube-controller-manager-amd64   "kube-controller-m..."   5 minutes ago       Up 5 minutes                            k8s_kube-controller-manager_kube-controller-manager-x1_kube-system_141e6bd8393d0c2ff62e471ef1ef1ff3_0
713b0d4ef0d6        gcr.io/google_containers/kube-scheduler-amd64            "kube-scheduler --..."   5 minutes ago       Up 5 minutes                            k8s_kube-scheduler_kube-scheduler-x1_kube-system_15e710c332c0a5bb569fd7371e1a2a04_0
e0b348678061        gcr.io/google_containers/kube-apiserver-amd64            "kube-apiserver --..."   5 minutes ago       Up 5 minutes                            k8s_kube-apiserver_kube-apiserver-x1_kube-system_afea2827f3bfa23dc32fedabc745119b_0
fdb592a2d0da        0a951668696f                                             "/opt/kube-addons.sh"    5 minutes ago       Up 5 minutes                            k8s_kube-addon-manager_kube-addon-manager-x1_kube-system_9831e93c3188555873fdb49f43198eef_0
9160db865efb        gcr.io/google_containers/etcd-amd64                      "etcd --listen-cli..."   5 minutes ago       Up 5 minutes                            k8s_etcd_etcd-x1_kube-system_baa35922a2db9b50e1760bd4574d15b6_2
01bb3b5248a0        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 5 minutes ago       Up 5 minutes                            k8s_POD_kube-controller-manager-x1_kube-system_141e6bd8393d0c2ff62e471ef1ef1ff3_0
877518f662c4        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 5 minutes ago       Up 5 minutes                            k8s_POD_kube-apiserver-x1_kube-system_afea2827f3bfa23dc32fedabc745119b_0
406a9e4da3a7        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 5 minutes ago       Up 5 minutes                            k8s_POD_kube-scheduler-x1_kube-system_15e710c332c0a5bb569fd7371e1a2a04_0
37830e95f46f        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 5 minutes ago       Up 5 minutes                            k8s_POD_etcd-x1_kube-system_baa35922a2db9b50e1760bd4574d15b6_2
1cdc843d7a2f        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 5 minutes ago       Up 5 minutes                            k8s_POD_kube-addon-manager-x1_kube-system_9831e93c3188555873fdb49f43198eef_0

What you expected to happen: The cluster should come up.

How to reproduce it (as minimally and precisely as possible): sudo CHANGE_MINIKUBE_NONE_USER=true minikube --bootstrapper kubeadm start --vm-driver=none

Output of minikube logs (if applicable): There are no logs for this run, only older entries.

[r2d4]

Are you running on a systemd system?

[webwurst]

Yes.

[jcmcken]

Running into this issue as well. It seems as if kubeadm is generating certs for 127.0.0.1, the API server ingress port, and some others, but not the actual IP address of the host. This leads to a TLS verification error, which prevents a bunch of things from starting up correctly.

[r2d4]

@jcmcken this is fixed in https://github.com/kubernetes/minikube/pull/2366

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[yaserrahimi]

I have this error
how can fixed it?

[landpy]

@yaserrahimi
I have the similar issue on my mac.
You can try to run "minikube start --bootstrapper localkube", but localkube is deprecated...

Success:
minikube start --bootstrapper localkube
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
Failed:
minikube start --bootstrapper kubeadm
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E0423 08:51:10.955625 727 start.go:281] Error restarting cluster: restarting kube-proxy: waiting for kube-proxy to be up for configmap update: timed out waiting for the condition
E0423 08:51:18.587221 727 util.go:151] Error uploading error message: : Post https://clouderrorreporting.googleapis.com/v1beta1/projects/k8s-minikube/events:report?key=AIzaSyACUwzG0dEPcl-eOgpDKnyKoUFgHdfoFuA: read tcp 192.168.3.2:51387->216.58.200.42:443: read: connection reset by peer

[TheBeachMaster]

@landpy : Thanks for the solution
minikube start --bootstrapper localkube --vm-driver=no worked in my case (not using any VM drivers)

[iwin4aids]

Version 0.26 just set localkube to deprecated and default will using kubeadm as bootstrapper.
I wonder how can i use the kubeadm to install?

if i add "--bootstrapper localkube" arg, it will be ok just like this(but with warnings):

minikube start --bootstrapper localkube

Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Downloading localkube binary
173.54 MB / 173.54 MB [============================================] 100.00% 0s
0 B / 65 B [----------------------------------------------------------] 0.00%
65 B / 65 B [======================================================] 100.00% 0sSetting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.

[adelowo]

Thanks @landy

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close