Minikube: HTTP_PROXY set: error getting Pods with label selector "k8s-app=kube-proxy" ... kube-proxy: Service Unavailable

Created on 16 Apr 2018 · 17Comments · Source: kubernetes/minikube

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Please provide the following details:
minikube start not listen port 8443

kubectl cluster-info
Kubernetes master is running at https://192.168.99.106:8443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Unable to connect to the server: Service Unavailable

Environment:

Minikube version (use minikube version): 0.26.0

OS (e.g. from /etc/os-release): macOS 10.13.4
VM Driver: vbox
ISO version : minikube-v0.26.0.iso
Install tools: homebrew
Others:
The above can be generated in one go with the following commands (can be copied and pasted directly into your terminal):

hongbin@MBP ~> kubectl cluster-info
Kubernetes master is running at https://192.168.99.106:8443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Unable to connect to the server: Service Unavailable

hongbin@MBP ~> minikube start  --docker-env HTTP_PROXY=http://<myproxy>:21089 --docker-env HTTPS_PROXY=https://<myproxy>:21089 --docker-env no_proxy=192.168.99.0/24 --v=9 --logtostderr
...

Setting up kubeconfig...
I0416 15:57:31.012657   32868 config.go:101] Using kubeconfig:  /Users/hongbin/.kube/config
Starting cluster components...
I0416 15:57:31.015960   32868 ssh_runner.go:58] Run:
sudo kubeadm alpha phase certs all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase kubeconfig all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase controlplane all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase etcd local --config /var/lib/kubeadm.yaml

I0416 15:57:34.998529   32868 loader.go:357] Config loaded from file /Users/hongbin/.kube/config
I0416 15:57:35.002515   32868 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: minikube/v1.10.0 (darwin/amd64) kubernetes/fc32d2f" https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy
I0416 15:57:36.008773   32868 round_trippers.go:405] GET https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy  in 1006 milliseconds
I0416 15:57:36.008798   32868 round_trippers.go:411] Response Headers:
I0416 15:57:36.008874   32868 kubernetes.go:119] error getting Pods with label selector "k8s-app=kube-proxy" [Get https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy: Service Unavailable]
I0416 15:57:36.510272   32868 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: minikube/v1.10.0 (darwin/amd64) kubernetes/fc32d2f" https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy
I0416 15:57:37.519401   32868 round_trippers.go:405] GET https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy  in 1009 milliseconds
I0416 15:57:37.519430   32868 round_trippers.go:411] Response Headers:
I0416 15:57:37.519470   32868 kubernetes.go:119] error getting Pods with label selector "k8s-app=kube-proxy" [Get https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy: Service Unavailable]
I0416 15:57:38.009508   32868 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: minikube/v1.10.0 (darwin/amd64) kubernetes/fc32d2f" https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy
I0416 15:57:39.014992   32868 round_trippers.go:405] GET https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy  in 1005 milliseconds
I0416 15:57:39.015020   32868 round_trippers.go:411] Response Headers:
I0416 15:57:39.015070   32868 kubernetes.go:119] error getting Pods with label selector "k8s-app=kube-proxy" [Get https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy: Service Unavailable]
I0416 15:57:39.510294   32868 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: minikube/v1.10.0 (darwin/amd64) kubernetes/fc32d2f" https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy
I0416 15:57:40.517726   32868 round_trippers.go:405] GET https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy  in 1007 milliseconds
I0416 15:57:40.517755   32868 round_trippers.go:411] Response Headers:
I0416 15:57:40.517796   32868 kubernetes.go:119] error getting Pods with label selector "k8s-app=kube-proxy" [Get https://192.168.99.106:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy: Service Unavailable]
I

enter minikube ssh
$ netstat -natl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.1:15441         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:42097           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 10.0.2.15:10010         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:59515           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:38463           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:18112           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:2049            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:37291           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:48235           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN
tcp        0   1656 10.0.2.15:22            10.0.2.2:54982          ESTABLISHED
tcp        0      0 10.0.2.15:44634         23.99.111.35:21089      ESTABLISHED
tcp        0      0 127.0.0.1:41266         127.0.0.1:15441         ESTABLISHED
tcp        0      0 127.0.0.1:15441         127.0.0.1:41266         ESTABLISHED
tcp        0      0 :::22                   :::*                    LISTEN
tcp        0      0 :::46661                :::*                    LISTEN
tcp        0      0 :::2376                 :::*                    LISTEN
tcp        0      0 :::10250                :::*                    LISTEN
tcp        0      0 :::5355                 :::*                    LISTEN
tcp        0      0 :::10255                :::*                    LISTEN
tcp        0      0 :::111                  :::*                    LISTEN

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Output of minikube logs (if applicable):

Anything else do we need to know:

arenetworking causfirewall-or-proxy eapiserver-refused help wanted kinbug prioritimportant-longterm

Source

kevinbin

👍26 😕8

Most helpful comment

@kevinbin I'm meeting the same issue, do you have any idea?

IlluYing on 19 Apr 2018

👍8 😕3

All 17 comments

@kevinbin I'm meeting the same issue, do you have any idea?

IlluYing on 19 Apr 2018

👍8 😕3

we have the same problems, any guys can help?

lexuszhi1990 on 14 May 2018

cute-angelia on 23 May 2018

+1 .(((((

sergeytkachenko on 2 Jun 2018

77720616 on 14 Jun 2018

emandere on 25 Jun 2018

Same here.

OS: Windows 10
VM Driver: Hyper-V
ISO version : minikube-v0.28.0.iso

$ netstat -natl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:43241           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:15441         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:39571           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 192.168.43.187:10010    0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:18112           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:2049            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:42307           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:59267           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:54467           0.0.0.0:*               LISTEN
tcp        0      1 192.168.43.187:33056    173.194.76.82:443       SYN_SENT
tcp        0      0 192.168.43.187:32862    173.194.76.82:443       TIME_WAIT
tcp        0      0 127.0.0.1:58354         127.0.0.1:15441         ESTABLISHED
tcp        0   1252 192.168.43.187:22       192.168.43.252:53678    ESTABLISHED
tcp        0      0 192.168.43.187:32908    173.194.76.82:443       TIME_WAIT
tcp        0      0 192.168.43.187:22       192.168.43.252:53020    ESTABLISHED
tcp        0      0 192.168.43.187:60490    173.194.76.82:443       TIME_WAIT
tcp        0      0 192.168.43.187:60654    173.194.76.82:443       TIME_WAIT
tcp        0      0 192.168.43.187:60614    173.194.76.82:443       TIME_WAIT
tcp        0      0 127.0.0.1:15441         127.0.0.1:58354         ESTABLISHED
tcp        0      0 :::2376                 :::*                    LISTEN
tcp        0      0 :::10250                :::*                    LISTEN
tcp        0      0 :::5355                 :::*                    LISTEN
tcp        0      0 :::10255                :::*                    LISTEN
tcp        0      0 :::111                  :::*                    LISTEN
tcp        0      0 :::22                   :::*                    LISTEN
tcp        0      0 :::34361                :::*                    LISTEN

I solved my problem. It was related to the internet connection of docker inside minikube
Here is my solution

pouyaesm on 5 Jul 2018

Was this ever fixed? I'm getting the same problem: minikube 0.28.0 with kube* 1.10.0 on mac as host and Virtualbox 5.2.20 r125813 (Qt5.6.3) as hypervisor.

notjames on 31 Oct 2018

Getting same problem in Windows 7, Kube* 1.10.0 and VirtualBox V 5.2.22 r126460 (Qt5.6.2). Anybody has any solution? Please update here.

paksboda on 22 Nov 2018

I don't think this has been fixed yet, at least it still does not work with VirtualBox, but I found a workaround that seems to work, just for clarity, here are my environment's brief specs:
VirtualBox version: 5.2.18
minikube version: v0.30.0
MacOS Mojave: version 10.14.1

What seems to work instead is to use different driver instead: https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#hyperkit-driver

$ curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-hyperkit \ && sudo install -o root -g wheel -m 4755 docker-machine-driver-hyperkit /usr/local/bin/

after running the above I was able to see:
/usr/local/bin/hyperkit -> /Applications/Docker.app/Contents/Resources/bin/com.docker.hyperkit

and I was able to start minikube:
$ minikube start --vm-driver=hyperkit

dgorczyca on 24 Nov 2018

I am getting same issue even with hyperkit.

minikube start --vm-driver=hyperkit --logtostderr -v=7

I1125 08:19:29.622240 27422 kubernetes.go:119] error getting Pods with label selector "k8s-app=kube-proxy" [Get https://192.168.64.2:8443/api/v1/namespaces/kube-system/pods?labelSelector=k8s-app%3Dkube-proxy: dial tcp 192.168.64.2:8443: connect: connection refused]

ernaveenverma on 25 Nov 2018

👍1

Related issues: #3145 and #3242

tstromberg on 24 Jan 2019

I've updated the HTTP proxy docs, which gives guidance to "no_proxy" values which will likely help with this: https://github.com/kubernetes/minikube/blob/master/docs/http_proxy.md

Does someone who is setting HTTP_*PROXY mind trying this with minikube v0.34.1, and include the output of minikube logs if they run into a failure?

tstromberg on 20 Feb 2019

Still got the same error with minikube v0.34.1, any idea ???

kuchaguangjie on 27 Feb 2019

With v0.34.1, removing ~/.minikube, first, then delete & start again seems to fix it.
And, backup & restore ~/.minikube/cache by hand, would be helpful to avoid download the caches again.

Refer: https://stackoverflow.com/questions/54903740/minikube-start-get-error-k8s-app-kube-proxy-connection-refused

kuchaguangjie on 27 Feb 2019

I believe this issue was resolved in the v1.1.0 release. Please try upgrading to the latest release of minikube, and if the same issue occurs, please re-open this bug. Thank you opening this bug report, and for your patience!

tstromberg on 22 May 2019

mark