Minikube: start hangs when --extra-config=apiserver.ServiceNodePortRange set (kube-apiserver CrashLoopBackOff)

[IBMRob]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

Please provide the following details:

Environment:

Minikube version (use minikube version):

• OS : Ubuntu 16.04
• VM Driver: virtualbox
• ISO version : minikube-v0.28.0
• Install tools: Completly clean OS install.
• Others:

What happened:

Starting minikube with minikube start --cpus 4 --disk-size 60g --memory 6144 --extra-config=apiserver.ServiceNodePortRange=80-32000 -v 10 hangs at Starting cluster components

What you expected to happen:

Minikube start up.

How to reproduce it (as minimally and precisely as possible):

Create VM with Ubuntu 16.04 LTS
Update VM to with latest patches
Install VirtualBox
Install Kubectl
Install Minikube
Start Minikube

Output of minikube logs (if applicable):

....
Jul 05 09:14:59 minikube kubelet[2776]: E0705 09:14:59.495253    2776 pod_workers.go:186] Error syncing pod 621213a6f3059728c13bfa6df5aa07d3 ("kube-apiserver-minikube_kube-system(621213a6f3059728c13bfa6df5aa07d3)"), skipping: failed to "StartContainer" for "kube-apiserver" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-minikube_kube-system(621213a6f3059728c13bfa6df5aa07d3)"
Jul 05 09:14:59 minikube kubelet[2776]: E0705 09:14:59.813319    2776 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.99.100:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.99.100:8443: getsockopt: connection refused
Jul 05 09:14:59 minikube kubelet[2776]: E0705 09:14:59.868084    2776 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.99.100:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.99.100:8443: getsockopt: connection refused
Jul 05 09:14:59 minikube kubelet[2776]: E0705 09:14:59.896238    2776 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.99.100:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.99.100:8443: getsockopt: connection refused

[IBMRob]

Note this is exactly the same error I see when running it natively on my MacoS machine as well. Same version of Minikube also hangs at Starting cluster components

[abdennour]

Same issue with my MacOS when specifying "memory" and "cpus" options: minikube --memory 8192 --cpus 2 start.
However, it works fine with a simple start minikube start

[scheung38]

Not really, with even the latest Mac High Sierra with

$ minikube start also takes ages

Sometimes after 20 mins it will connect, sometimes after 30 mins it will, then sometimes after 30 mins hangs....

Mac 10.13.6
brew cask install minikube
minikube version: v0.28.1

Tried

minikube delete
rm -rf ~/.minikube

before

minikube start

but it is highly unpredictable what it will do...

Can anyone confirm what it is right solution to launch efficiently?

[aclowkey]

Getting the same issue on Ubuntu..Deleting and restarting doesn't work..
just running minikube start

[kaelzhang]

Encounter this problem every time when I upgrade, reinstall or even slightly touch minikube !!

Really helpless with the error.

Is there any official instructions about this problem, and more hints to the tty to tell us what to do next?

[shimron]

same error with minikube v0.28.2 on macOS 10.13.6

[geropl]

For me (Ubuntu 16.04, minikube v0.28.2, kubectl v1.11.0) leaving out --extra-config=apiserver.ServiceNodePortRange did the trick.

minikube start works but minikube start --extra-config=apiserver.ServiceNodePortRange=80-32767 does not.

UPDATE:
After inspecting the docker logs of the failing apiserver with minikube ssh, docker ps -n 10 and docker logs -t I found this little gem:

2018-09-15T12:15:52.225166559Z       --service-node-port-range portRange                       A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
[...]
2018-09-15T12:15:52.225219539Z error: unknown flag: --ServiceNodePortRange

It turned out the name of the config apiserver.ServiceNodePortRange was changed to apiserver.service-node-port-range...

With that option the following combination works for me:
Ubuntu 16.04, minikube v0.28.2, kubectl v1.11.0 and minikube start --extra-config=apiserver.service-node-port-range=80-32767 --kubernetes-version=v1.10.0.

[mkm177]

I am having the same issue on MacOS High Sierra (v10.13.6).
minikube 0.28.2, Kubectl v1.11.0.
Neither minikube start, nor minikube start ----vm-driver=virtualbox work.

Heres the latest error message after about a 5 minute wait:

$ minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E0914 12:55:06.040664    1434 start.go:299] Error restarting cluster:  restarting kube-proxy: waiting for kube-proxy to be up for configmap update: timed out waiting for the condition

[owenallenaz]

Switching to --extra-config=apiserver.service-node-port-range=80-32767 worked for me. Thanks for the tip!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[tstromberg]

Closing in preference to the more general issue of invalid apiserver flags causing havoc: #3532

[k8s-ci-robot]

@maktas: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[maktas]

I am having the same issue on Windows 10 (full version: 10.0.18362 N/A Build 18362).
minikube v1.12.1, Kubectl: v1.18.3
Neither minikube start, nor minikube start ----vm-driver=virtualbox work.

After several failure messages like "failed to "StartContainer" for "kube-apiserver" with CrashLoopBackOff", minikube start commands eventually fail with: "Unfortunately, an error has occurred: timed out waiting for the condition" message.