minikube start hung: etcd CrashLoopBackoff

Created on 24 Apr 2018  ·  65Comments  ·  Source: kubernetes/minikube

BUG REPORT

Environment:

Minikube version
v0.26.1

  • OS
    MacOS High Sierra 10.13.4

  • VM Driver
    Virtual Box

  • ISO version
    Latest as of today

  • Install tools:

  • Others:

What happened:

Initial minikube start hangs at 'Starting cluster components...' and never completes. Note that the vm 'is' running, and I can log into it from the Virtual Box console. From another terminal window, 'minikube ip' does return an ip address.

$ minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubeadm v1.10.0
Downloading kubelet v1.10.0
Finished Downloading kubelet v1.10.0
Finished Downloading kubeadm v1.10.0
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...

What you expected to happen:

Ummm - for it to complete, of course...

How to reproduce it (as minimally and precisely as possible):

minikube start

Output of minikube logs (if applicable):

(attached)
minikube-logs-output.txt

cetcd cvirtualbox eCrashLoopBackOff ehung-start kinbug omacos
Source
picture vinceskahan
👍27 ❤3

Most helpful comment

Temporary workaround:

Using these instructions as an example, I forced an install of 0.25.2 in the meantime:

minikube stop
minikube delete
brew cask uninstall --force minikube
brew cask edit minikube
#See below for file
brew cask install minikube

Using this file:

cask 'minikube' do
  version '0.25.2'
  sha256 'dc5b00c4a06e8160bd607732c9a2294598d803716e353293b4463cc2c9539eec'

  # storage.googleapis.com/minikube was verified as official when first introduced to the cask
  url "https://storage.googleapis.com/minikube/releases/v#{version}/minikube-darwin-amd64"
  appcast 'https://github.com/kubernetes/minikube/releases.atom',
          checkpoint: 'dc5b00c4a06e8160bd607732c9a2294598d803716e353293b4463cc2c9539eec'
  name 'Minikube'
  homepage 'https://github.com/kubernetes/minikube'

  depends_on formula: 'kubernetes-cli'
  container type: :naked

  binary 'minikube-darwin-amd64', target: 'minikube'

  zap trash: '~/.minikube'
end
picture ttripp on 1 May 2018
👍15 🎉3

All 65 comments

I have the same issue except that I am using the hyperkit driver, and I'm installing a k8s 1.9.0 cluster. Gets stuck at "Starting cluster components"

This is on MacOS High Sierra (10.13.3)

davidnewman picture davidnewman on 24 Apr 2018

Same issue for me.

mgdevstack picture mgdevstack on 24 Apr 2018

If you have an old ~/.minikube, try

$ minikube delete
$ rm -rf ~/.minikube

Try starting up again.

eldada picture eldada on 24 Apr 2018
👍9 🎉5 ❤1

same issue here

Also try minikube delete && rm -rf ~/.minikube

But it not worked

Use minikube logs -f and see the error is:

Apr 24 15:44:59 minikube kubelet[3287]: E0424 15:44:59.532988 3287 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.99.100:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.99.100:8443: getsockopt: connection refused
Apr 24 15:44:59 minikube kubelet[3287]: E0424 15:44:59.533844 3287 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.99.100:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.99.100:8443: getsockopt: connection refused

playniuniu picture playniuniu on 24 Apr 2018
👍17

Similar problem on minikube for windows. I did a minikube delete && rm ~/.minikube and reinstalled v0.25.2 and minikube started in <30 seconds.

slmagus picture slmagus on 25 Apr 2018
👍6

@slmagus v0.25.2 is running well as expected. But seems there is an issue #2753 with 0.26.0 and 0.26.1. Since last 10 days, I'm facing issue, before that 0.26.* were working.

mgdevstack picture mgdevstack on 25 Apr 2018
👍2

Same issue on Linux Mint (ubuntu derived distro).

cliveseldon picture cliveseldon on 26 Apr 2018

Same issue with me : Apr 26 21:37:10 minikube kubelet[3154]: E0426 21:37:10.657560 3154 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://192.168.64.22:8443/api/v1/nodes: dial tcp 192.168.64.22:8443: getsockopt: connection refused
Apr 26 21:37:11 minikube kubelet[3154]: E0426 21:37:11.445468 3154 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.22:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.22:8443: getsockopt: connection refused
Apr 26 21:37:11 minikube kubelet[3154]: E0426 21:37:11.446275 3154 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.22:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.22:8443: getsockopt: connection refused

lokeshmadan picture lokeshmadan on 26 Apr 2018

Same issue with same config as @vinceskahan also getting the getsockopt: connection refused errors although I do have some extra cmdline parms minikube start --cpus 4 --disk-size 60g --memory 6144 --vm-driver=virtualbox --extra-config=apiserver.ServiceNodePortRange=80-32000

IBMRob picture IBMRob on 27 Apr 2018

Same issue here, on MacOS w/ hyperkit driver, kubeadm bootstrapper, k8s version v1.9.6. minikube ssh does work, and using that I see that it appears that the kube-apiserver container keeps unexpectedly exiting.

$ minikube ssh 'docker ps -a'
CONTAINER ID        IMAGE                                                    COMMAND                  CREATED              STATUS                          PORTS               NAMES
601c48f3bfb8        gcr.io/google_containers/kube-apiserver-amd64            "kube-apiserver --..."   About a minute ago   Exited (2) About a minute ago                       k8s_kube-apiserver_kube-apiserver-minikube_kube-system_268917328163745905396fe3f66a8905_5
000fe7a44e2b        gcr.io/google_containers/etcd-amd64                      "etcd --listen-cli..."   4 minutes ago        Up 4 minutes                                        k8s_etcd_etcd-minikube_kube-system_d2c35e6cc76ad8c4c16b9f75c1f14a60_2
de44ce926d64        gcr.io/google_containers/kube-controller-manager-amd64   "kube-controller-m..."   4 minutes ago        Up 4 minutes                                        k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_ab782b037996db13fa5d60b54d0fdc0c_0
709611ac3ea6        k8s.gcr.io/kube-addon-manager                            "/opt/kube-addons.sh"    4 minutes ago        Up 4 minutes                                        k8s_kube-addon-manager_kube-addon-manager-minikube_kube-system_c9760ab157812e306964c796adcfaefb_1
70498c0aea0d        gcr.io/google_containers/kube-scheduler-amd64            "kube-scheduler --..."   4 minutes ago        Up 4 minutes                                        k8s_kube-scheduler_kube-scheduler-minikube_kube-system_0f95caae4a7fffab092dfa6db3c27347_3
a3f9ff1d712d        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 4 minutes ago        Up 4 minutes                                        k8s_POD_etcd-minikube_kube-system_d2c35e6cc76ad8c4c16b9f75c1f14a60_2
0b12b0502878        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 4 minutes ago        Up 4 minutes                                        k8s_POD_kube-scheduler-minikube_kube-system_0f95caae4a7fffab092dfa6db3c27347_3
acc562ef0a02        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 4 minutes ago        Up 4 minutes                                        k8s_POD_kube-addon-manager-minikube_kube-system_c9760ab157812e306964c796adcfaefb_1
ade51feb9df4        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 4 minutes ago        Up 4 minutes                                        k8s_POD_kube-controller-manager-minikube_kube-system_ab782b037996db13fa5d60b54d0fdc0c_0
aed6f830a8ef        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 4 minutes ago        Up 4 minutes                                        k8s_POD_kube-apiserver-minikube_kube-system_268917328163745905396fe3f66a8905_0

Looking into it a bit farther, it looks like --require-kubeconfig is being passed to kube-apiserver but the flag is unknown/unsupported.

$ docker logs 601c48f3bfb8
unknown flag: --require-kubeconfig
# REMOVED THE REST OF THE OUTPUT #

Updating the k8s version to v1.10.2 appears to fix the issue for me, but I'm sure that's not really much of a fix for many people.

TLDR looks like minikube is passing unsupported flags to _some_ versions of the k8s API server.

Here's a reference link re: the --require-kubeconfig flag and its recent deprecation/removal.

alanbrent picture alanbrent on 27 Apr 2018

just to share that on minikube version: v0.26.1, --extra-config=apiserver.service-node-port-range=90-32000 worked for me.

p.s. the minikube logs -f helped me identify what the problem was.

gsaslis picture gsaslis on 29 Apr 2018
👍2

update - the '--extra-config' that gsaslis mentioned above did 'not' work for me. Total hang to the point where the computer almost totally locked up and I had to reset it.

Downgrading minikube to 0.25.2 'did' work just fine, so the problem appears to be something in the 0.26.x codebase.

vinceskahan picture vinceskahan on 29 Apr 2018
👍3

I have the same issue with hyperkit v0.20171204-60-g0e5b6b driver and MacOS High Sierra 10.13.4. In minikube logs got:
Apr 30 11:37:55 minikube kubelet[2977]: E0430 11:37:55.301205 2977 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://192.168.64.100:8443/api/v1/nodes: dial tcp 192.168.64.100:8443: getsockopt: connection refused Apr 30 11:37:55 minikube kubelet[2977]: E0430 11:37:55.570213 2977 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://192.168.64.100:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.100:8443: getsockopt: connection refused Apr 30 11:37:55 minikube kubelet[2977]: E0430 11:37:55.572007 2977 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.100:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.100:8443: getsockopt: connection refused Apr 30 11:37:55 minikube kubelet[2977]: E0430 11:37:55.572993 2977 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://192.168.64.100:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.100:8443: getsockopt: connection refused

Neither removing .minikube directory nor --extra-config=apiserver.service-node-port-range=90-32000 works for me

dpacierpnik picture dpacierpnik on 30 Apr 2018

I'm pretty sure the issue is in this line, which was added as part of this commit in the lead up to the 0.26 release.

AFAICT, the --require-kubeconfig flag stopped working in 1.9, so what we'd need is to only enable it for versions less than 1.9.0-alpha.0.

alanbrent picture alanbrent on 1 May 2018

You _can_ work around this by using the localkube bootstrapper.

I hope a fix for this is forthcoming soon.

krancour picture krancour on 1 May 2018
👍1

Temporary workaround:

Using these instructions as an example, I forced an install of 0.25.2 in the meantime:

minikube stop
minikube delete
brew cask uninstall --force minikube
brew cask edit minikube
#See below for file
brew cask install minikube

Using this file:

cask 'minikube' do
  version '0.25.2'
  sha256 'dc5b00c4a06e8160bd607732c9a2294598d803716e353293b4463cc2c9539eec'

  # storage.googleapis.com/minikube was verified as official when first introduced to the cask
  url "https://storage.googleapis.com/minikube/releases/v#{version}/minikube-darwin-amd64"
  appcast 'https://github.com/kubernetes/minikube/releases.atom',
          checkpoint: 'dc5b00c4a06e8160bd607732c9a2294598d803716e353293b4463cc2c9539eec'
  name 'Minikube'
  homepage 'https://github.com/kubernetes/minikube'

  depends_on formula: 'kubernetes-cli'
  container type: :naked

  binary 'minikube-darwin-amd64', target: 'minikube'

  zap trash: '~/.minikube'
end
ttripp picture ttripp on 1 May 2018
👍15 🎉3

The fix still isn't clear to me. What versions of Kubernetes is this reproducing with?

It looks to me like the require-kubeconfig flag was only removed for 1.10, and should still have been present in 1.9:
https://github.com/kubernetes/kubernetes/pull/58367

dlorenc picture dlorenc on 1 May 2018

This happens on Linux (Fedora), not just mac and seems to be a problem with versions > 0.26.x...

Cryptophobia picture Cryptophobia on 2 May 2018

As @dlorenc said it looks like bug caused by removing the require-kubeconfig flag. It worked perfectly fine v1.10.0

reghuramrv picture reghuramrv on 3 May 2018

This happens on Linux (Fedora), not just mac and seems to be a problem with versions > 0.26.x...

Which versions of Kubernetes (not minikube) are you seeing this with?

dlorenc picture dlorenc on 3 May 2018

@dlorenc

Which versions of Kubernetes (not minikube) are you seeing this with?

I was seeing this when trying to run k8s v1.9.4 and v1.9.7 locally. I had to downgrade to 0.25.2 in order to make it work.

Cryptophobia picture Cryptophobia on 3 May 2018

same issues , failed to run minikube start，minikube version: v0.26.1 ， OSX

➜  ~ minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
ChinaSilence picture ChinaSilence on 4 May 2018 
$ minikube version
minikube version: v0.26.1

$ minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubelet v1.10.0
Downloading kubeadm v1.10.0
Finished Downloading kubelet v1.10.0
Finished Downloading kubeadm v1.10.0
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E0504 15:02:46.676882   24473 start.go:281] Error restarting cluster:  restarting kube-proxy: waiting for kube-proxy to be up for configmap update: timed out waiting for the condition

$ minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100

$ minikube stop
Stopping local Kubernetes cluster...
Machine stopped.

$ minikube start
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
[hang]
bitfield picture bitfield on 4 May 2018

try to run minikube start --bootstrapper=localkube , it works well...

➜  ~ minikube start --bootstrapper=localkube
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
ChinaSilence picture ChinaSilence on 5 May 2018
👍15

Doesn't seem to be OS specific as I've experienced this on Mint, Ubuntu and Windows 10 Home.

@ChinaSilence setting the bootstrapper to localkube does indeed work on Windows at least (can't check linux just now but I assume it works there too), but I get this warning:

WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper,

DickChesterwood picture DickChesterwood on 5 May 2018

Using minikube start --bootstrapper=localkube on OSX made no difference. In my case

~❯ minikube start --bootstrapper=localkube
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Downloading Minikube ISO
 150.53 MB / 150.53 MB [============================================] 100.00% 0s
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Downloading localkube binary
 173.54 MB / 173.54 MB [============================================] 100.00% 0s
 0 B / 65 B [----------------------------------------------------------]   0.00%
 65 B / 65 B [======================================================] 100.00% 0sSetting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
~❯ minikube status
minikube: Running
cluster: Stopped
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100

I can never get the cluster to run.

memelet picture memelet on 5 May 2018

Thanks @ChinaSilence
minikube start --bootstrapper=localkube helps me start cluster components!

$ kubectl cluster-info

Kubernetes master is running at https://192.168.99.100:8443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Liwink picture Liwink on 6 May 2018
👍3

As @Liwink said, I can confirm that using --bootstrapper=localkube helps to work around this issue. :ok_hand:

Depado picture Depado on 7 May 2018

This workaround worked:

minikube start --bootstrapper=localkube

Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Downloading localkube binary
 173.54 MB / 173.54 MB [============================================] 100.00% 0s
 0 B / 65 B [----------------------------------------------------------]   0.00%
 65 B / 65 B [======================================================] 100.00% 0sSetting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
luisnuxx picture luisnuxx on 8 May 2018
👍8 🎉2

This workaround worked:

{10:41}~ ➭ minikube stop
Stopping local Kubernetes cluster...
Machine stopped.
{10:41}~ ➭ minikube start --bootstrapper=localkube
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Downloading localkube binary
 173.54 MB / 173.54 MB [============================================] 100.00% 0s
 0 B / 65 B [----------------------------------------------------------]   0.00%
 65 B / 65 B [======================================================] 100.00% 0sSetting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
{10:43}~ ➭ kubectl get nodes
NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     <none>    44s       v1.10.0

But It have a WARNING.

WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]

So，I hope it would be fix.

yangwenmai picture yangwenmai on 10 May 2018

+1

chinajuanbob picture chinajuanbob on 14 May 2018

+1

Update: @ttripp workaround worked for me !!!, the --bootstrapper=localkube didn't worked because minikube status gave me cluster: stoped ...

for now @ttripp thank you man !

DavidNorena picture DavidNorena on 14 May 2018
👍1

confirmed 0.27.0 fixed this issue for me.

$ minikube start --vm-driver=hyperkit
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Downloading Minikube ISO
 150.53 MB / 150.53 MB [============================================] 100.00% 0s
Getting VM IP address...
Moving files into cluster...
Downloading kubelet v1.10.0
Downloading kubeadm v1.10.0
Finished Downloading kubelet v1.10.0
Finished Downloading kubeadm v1.10.0
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.
jostyee picture jostyee on 17 May 2018

I can also get minikube up and running. But cannot start the dashboard anymore with 0.26

~/si❯ minikube dashboard
Could not find finalized endpoint being pointed to by kubernetes-dashboard: Error validating service: Error getting service kubernetes-dashboard: services "kubernetes-dashboard" not found
memelet picture memelet on 17 May 2018

I also see that I have no dns running. Is that normal?

~/si❯ kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY     STATUS              RESTARTS   AGE
default       hello-minikube-6c47c66d8-t4nf6   0/1       ContainerCreating   0          9m
kube-system   kube-addon-manager-minikube      0/1       ContainerCreating   0          11m
memelet picture memelet on 17 May 2018

@memelet your minikube logs is?

Robot684 picture Robot684 on 17 May 2018

@Robot684 I'll have to get back in a bit on that. I uninstalled minikube and now cannot install. I assume this temporary.

~/si❯ brew cask install minikube
==> Satisfying dependencies
All Formula dependencies satisfied.
==> Downloading https://storage.googleapis.com/minikube/releases/v0.26.1/minikube-darwin-amd64

curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to storage.googleapis.com:443
Error: Download failed on Cask 'minikube' with message: Download failed: https://storage.googleapis.com/minikube/releases/v0.26.1/minikube-darwin-amd64

Hmm, but I can download the package directly without using brew...

memelet picture memelet on 17 May 2018

@memelet

For the dashboard

$ kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

Taken from here

DavidNorena picture DavidNorena on 17 May 2018

Because 0.27.0 hangs too, although 0.25.2 continues to function if I use
that.
kubectl is 1.10.2 if that matters.

Macbook air running macOS High Sierra 10.13.4

On Thu, May 17, 2018 at 9:17 AM, Robot684 notifications@github.com wrote:

@memelet https://github.com/memelet Why not use the latest version?
curl -Lo minikube https://storage.googleapis.
com/minikube/releases/v0.27.0/minikube-darwin-amd64 && chmod +x minikube
&& sudo mv minikube /usr/local/bin/

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/minikube/issues/2765#issuecomment-389924190,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAk2KtCJo2PBpPm_gENI5SLreYtQER6qks5tzaKXgaJpZM4TgtKZ
.

--
----- [email protected] ----

vinceskahan picture vinceskahan on 17 May 2018

@DavidNorena Yes that is how I installed the dashboard. It failed to start. (Need to reinstall to get the error...)

But I also of course tried minikube dashboard which worked with 0.25.

I have also tried to downgrade to 0.25 and that did not work anymore. So maybe its more than just the version for me.

memelet picture memelet on 17 May 2018

@vinceskahan v0.25 is work well?

Robot684 picture Robot684 on 18 May 2018

Minikube 0.27 also doesn't work for me on mac osx 10.13.4

But when use minikube start --vm-driver=hyperkit --bootstrapper=localkube it works

playniuniu picture playniuniu on 18 May 2018

@playniuniu 使用VirtualBox是不是不行啊

Robot684 picture Robot684 on 18 May 2018

@Robot684 没有试过，应该可以吧

playniuniu picture playniuniu on 18 May 2018

@playniuniu 不行，我测试了从0.26以上的版本都不可以，我现在在测试你说的启动方式

Robot684 picture Robot684 on 18 May 2018

@playniuniu 我试了下，为什么cluster是关闭的呢？

`
➜ ~ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready 1m v1.10.0

➜ ~ minikube status
minikube: Running
cluster: Stopped
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.64.3
`

Robot684 picture Robot684 on 18 May 2018

@Robot684 我的也是，应该不影响使用吧

playniuniu picture playniuniu on 18 May 2018

@playniuniu 你可以进行后续的模拟创建pod测试吗？

Robot684 picture Robot684 on 18 May 2018

@playniuniu / @Robot684 please note that when you write a comment on a GitHub issue (which has so far 25+ participants), by default, a notification email is sent to each and everyone of them.

when your comment is not in english, you are essentially sending us messages in your own language, which we all can't understand. that doesn't feel very nice.

please reconsider using just English. ;)

gsaslis picture gsaslis on 18 May 2018

same problem on macOS High Sierra 10.13.4 with minikube v0.27.0

maybe a different cause

log:

May 19 15:59:00 minikube kubelet[2893]: I0519 15:59:00.888564    2893 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
May 19 15:59:00 minikube kubelet[2893]: I0519 15:59:00.893989    2893 kubelet_node_status.go:82] Attempting to register node minikube
May 19 15:59:00 minikube kubelet[2893]: E0519 15:59:00.894672    2893 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://192.168.64.2:8443/api/v1/nodes: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.023553    2893 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.023654    2893 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "kube-apiserver-minikube_kube-system(6c3bcd23aafaaad42909185679ecdb62)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.023696    2893 kuberuntime_manager.go:646] createPodSandbox for pod "kube-apiserver-minikube_kube-system(6c3bcd23aafaaad42909185679ecdb62)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.023865    2893 pod_workers.go:186] Error syncing pod 6c3bcd23aafaaad42909185679ecdb62 ("kube-apiserver-minikube_kube-system(6c3bcd23aafaaad42909185679ecdb62)"), skipping: failed to "CreatePodSandbox" for "kube-apiserver-minikube_kube-system(6c3bcd23aafaaad42909185679ecdb62)" with CreatePodSandboxError: "CreatePodSandbox for pod \"kube-apiserver-minikube_kube-system(6c3bcd23aafaaad42909185679ecdb62)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause-amd64:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.033510    2893 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.033594    2893 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "etcd-minikube_kube-system(41069a7dedf9b86c8d5dd0a50a0ac538)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.033616    2893 kuberuntime_manager.go:646] createPodSandbox for pod "etcd-minikube_kube-system(41069a7dedf9b86c8d5dd0a50a0ac538)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.033736    2893 pod_workers.go:186] Error syncing pod 41069a7dedf9b86c8d5dd0a50a0ac538 ("etcd-minikube_kube-system(41069a7dedf9b86c8d5dd0a50a0ac538)"), skipping: failed to "CreatePodSandbox" for "etcd-minikube_kube-system(41069a7dedf9b86c8d5dd0a50a0ac538)" with CreatePodSandboxError: "CreatePodSandbox for pod \"etcd-minikube_kube-system(41069a7dedf9b86c8d5dd0a50a0ac538)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause-amd64:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.410607    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.410629    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:01 minikube kubelet[2893]: E0519 15:59:01.411414    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:02 minikube kubelet[2893]: E0519 15:59:02.414657    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:02 minikube kubelet[2893]: E0519 15:59:02.414683    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:02 minikube kubelet[2893]: E0519 15:59:02.416531    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:02 minikube kubelet[2893]: I0519 15:59:02.699420    2893 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
May 19 15:59:03 minikube kubelet[2893]: E0519 15:59:03.416798    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:03 minikube kubelet[2893]: E0519 15:59:03.418399    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:03 minikube kubelet[2893]: E0519 15:59:03.420191    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:04 minikube kubelet[2893]: E0519 15:59:04.424624    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:04 minikube kubelet[2893]: E0519 15:59:04.424757    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:04 minikube kubelet[2893]: E0519 15:59:04.424864    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:05 minikube kubelet[2893]: E0519 15:59:05.431137    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:05 minikube kubelet[2893]: E0519 15:59:05.431632    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:05 minikube kubelet[2893]: E0519 15:59:05.431799    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:05 minikube kubelet[2893]: E0519 15:59:05.687566    2893 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found
May 19 15:59:06 minikube kubelet[2893]: E0519 15:59:06.434120    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.2:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:06 minikube kubelet[2893]: E0519 15:59:06.435347    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.2:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused
May 19 15:59:06 minikube kubelet[2893]: E0519 15:59:06.436853    2893 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.2:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.2:8443: getsockopt: connection refused

problem solved with __localkube__

minikube start --vm-driver=hyperkit --boostrapper=localkube
Hujun picture Hujun on 19 May 2018

I too am encountering the same set of problems :-

Configuration

  • Minikube: Both 0.26.1 as well as 0.27
  • OS: osx 10.13.4

Issues:

See attempts below that did not work :-

minikube start --vm-driver=hyperkit --bootstrapper=localkube

$> kubectl get all --all-namespaces
NAMESPACE     NAME                              READY     STATUS              RESTARTS   AGE
kube-system   pod/kube-addon-manager-minikube   0/1       ContainerCreating   0          8m

NAMESPACE   NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
default     service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   8m

minikube start

May 19 19:57:08 minikube kubelet[2986]: E0519 19:57:08.559046    2986 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.5:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.5:8443: getsockopt: connection refused
May 19 19:57:09 minikube kubelet[2986]: E0519 19:57:09.561264    2986 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://192.168.64.5:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.5:8443: getsockopt: connection refused
May 19 19:57:09 minikube kubelet[2986]: E0519 19:57:09.561264    2986 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://192.168.64.5:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 192.168.64.5:8443: getsockopt: connection refused
May 19 19:57:09 minikube kubelet[2986]: E0519 19:57:09.561585    2986 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://192.168.64.5:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 192.168.64.5:8443: getsockopt: connection refused
May 19 19:57:09 minikube kubelet[2986]: E0519 19:57:09.736862    2986 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found
vinaynair picture vinaynair on 19 May 2018
👍3

+1

pstinghua picture pstinghua on 20 May 2018

Still hanging for me - mac, linux and windows - on the new 0.27 (unless the deprecated --bootstrapper=localkube is applied)

DickChesterwood picture DickChesterwood on 21 May 2018

@vinaynair Maybe you cannot pull the images, try to run kubectl get events -n kube-system get more details

ChinaSilence picture ChinaSilence on 22 May 2018

Same here - High Sierra and had to use --bootstrapper=localkube to get a working cluster.

sc68cal picture sc68cal on 22 May 2018

Similar error as @vinaynair, I had thought it was my environment issue, while it seems a common one.

$ minikube start --vm-driver=hyperkit --bootstrapper=localkube
$ kubectl get po --all-namespaces
NAMESPACE     NAME                                READY     STATUS              RESTARTS   AGE
default       samplecontroller-5686f7649f-c2r42   0/1       ContainerCreating   0          13d
kube-system   kube-addon-manager-minikube         0/1       ContainerCreating   0          13d

$ kubectl get events -n kube-system

12d         12d          1         kube-scheduler-minikube.152d9942416d2a42            Pod                     Normal    SuccessfulMountVolume    kubelet, minikube    MountVolume.SetUp succeeded for volume "kubeconfig" 
12d         12d          346       kube-scheduler-minikube.152d99465a77c2d5            Pod                     Warning   FailedCreatePodSandBox   kubelet, minikube    Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
12d         12d          70        kube-scheduler-minikube.152d99f6e5e6fedf            Pod                     Warning   FailedCreatePodSandBox   kubelet, minikube    Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause-amd64:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 10.0.2.3:53: server misbehaving
ningning54321 picture ningning54321 on 24 May 2018

I think this was fixed with https://github.com/kubernetes/minikube/pull/2791

dlorenc picture dlorenc on 24 May 2018
👍1

It does not appear that fixed the issue, because I encountered this issue two days ago and only was able to solve it by using the workaround.

scolli572 » minikube version                                                                                                                                                                        1 ↵
minikube version: v0.27.0
sc68cal picture sc68cal on 24 May 2018

agree - this is 'not' fixed in 0.27.0 without using the workaround

vinceskahan picture vinceskahan on 24 May 2018

Ya, and localkube isn't really a workaround -- it's a deprecated bootstrapper. This needs to work with kubeadm as well.

arbourd picture arbourd on 24 May 2018

I think there might have been several different underlying issues being tracked in here. Would someone still experiencing this mind opening another issue with all the info from the template?

dlorenc picture dlorenc on 24 May 2018

I think closing an issue and asking a user to create a brand new issue from a template is .... perhaps not ideal.

sc68cal picture sc68cal on 24 May 2018

Unfortunately there's just not enough information in any of these comments beyond the initial ones (involving the --require-kubeconfig flag, which was fixed in the linked PR). I'm sure there are other issues, but the best way to help us find and fix them is to include the debugging information asked for in the template.

dlorenc picture dlorenc on 24 May 2018

@dlorenc - I have to call baloney on that one. Please read 'my' original problem description for this ticket at the very top. The expectation is that I can run "minikube start" per the docs, with no options or flags or workarounds and have the blasted thing actually start. If that's somehow unclear, I can't help you.

All the other comments, in my reading, are people mentioning:

  • it's broken for me too on (insert platform description here)
  • but it used to work in 0.25.2 for me too on (insert platform description here)
  • and it seems it can be worked around with version x.xx.x with this set of deprecated flags

It's broken if it used to work before. It's broken if deprecated workaround flags are required. So it's still broken for me (the original poster) on the same High Sierra Macbook that I reported it on.

And I 'did' try to use 0.27.0 and did verify it is still broken here on that platform.

vinceskahan picture vinceskahan on 24 May 2018

I'm going to lock this issue in the interest of keeping things productive. If you'd like to help see issues get fixed, please open a new issue with the output of "minikube info" and "minikube logs" attached. Without that we can't really debug anything.

dlorenc picture dlorenc on 24 May 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

vainikkaj picture vainikkaj  ·  3Comments
ahmetb picture ahmetb  ·  3Comments
alber70g picture alber70g  ·  3Comments
Starefossen picture Starefossen  ·  3Comments
StasPerekrestov picture StasPerekrestov  ·  3Comments