Minikube: Localkube >= v1.7.3 requires minikube > v0.21.0

[ssuarez6]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Minikube version (use minikube version):
minikube version: v0.20.0

Environment:

• OS (e.g. from /etc/os-release): NAME="Manjaro Linux" ID=manjaro PRETTY_NAME="Manjaro Linux" ANSI_COLOR="1;32" HOME_URL="https://www.manjaro.org/" SUPPORT_URL="https://www.manjaro.org/" BUG_REPORT_URL="https://bugs.manjaro.org/"

• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): "DriverName": "virtualbox"
• ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): v0.20.0
• Install tools:
• Others:

What happened:
First I started minikube and since that I'm trying to open the dashboard so I can configure my Pods and Services, but I just got this output: Could not find finalized endpoint being pointed to by kubernetes-dashboard: Error validating service: Error getting service kubernetes-dashboard: Get https://192.168.98.100:8443/api/v1/namespaces/kube-system/services/kubernetes-dashboard: dial tcp 192.168.98.100:8443: getsockopt: connection refused

What you expected to happen: To open a browser tab with the kubernetes dashboard

How to reproduce it (as minimally and precisely as possible): With my environment and using virtualbox 5.1.22r115126, try to start minikube and then type minikube dashboard

Anything else do we need to know:

[r2d4]

Can you post the output of some of these commands?

minikube status
minikube logs
kubectl get pods --all-namespaces

[ssuarez6]

Offcourse! Here:

• minikube status: minikube: Running localkube: Running kubectl: Correctly Configured: pointing to minikube-vm at 192.168.98.100

• minikube logs: logs.txt

• kubectl get pods --all-namespaces: The connection to the server 192.168.98.100:8443 was refused - did you specify the right host or port?

[spnzig]

You are not able to connect to the kubernetes server node. Could you try doing a kubectl version and paste the output here?

[ssuarez6]

Sure. kubectl version:
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.1", GitCommit:"1dc5c66f5dd61da08412a74221ecc79208c2165b", GitTreeState:"clean", BuildDate:"2017-07-14T02:00:46Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} The connection to the server 192.168.90.100:8443 was refused - did you specify the right host or port?

[spnzig]

Could you do a "sudo ifconfig vboxnet0 up" and try to recheck the kubectl version?

[ssuarez6]

After sudo ifconfig vboxnet0 up the output of kubectl version seems to be the same:
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.1", GitCommit:"1dc5c66f5dd61da08412a74221ecc79208c2165b", GitTreeState:"clean", BuildDate:"2017-07-14T02:00:46Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} The connection to the server 192.168.90.100:8443 was refused - did you specify the right host or port?

[spnzig]

I know this is a nasty suggestion. But is it possible for you to stop, delete and restart the minikube vm? Maybe removing some third party resources and a clean install would help. Maybe some minikube file is corrupted. Also i had few issues due to this. So a clean install came to my rescue.

[ssuarez6]

When doing that, now this error is shown:
VBoxManage: error: The virtual machine 'minikube' has terminated unexpectedly during startup with exit code 1 (0x1)

VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine

[spnzig]

Do a clean install. sudo rm -rf ~/.minikube , remove the binaries for minikube and kubectl from /usr/... and then re-install minikube and kubectl. This should work.

[ssuarez6]

I removed ~./minikube folder and also the /usr/bin/ binaries for minikube and kubectl-bin.
Re-installed using yaourt -S minikube and yaourt -S kubectl-bin. Minikube VM didn't start beacuse of the default given IP, and after changed it the VM started successfully. Anyway when trying to show the dashboard, still getting the Could not find finalized endpoint being pointed to by kubernetes-dashboard: Error validating service: Error getting service kubernetes-dashboard: Get https://192.168.90.100:8443/api/v1/namespaces/kube-system/services/kubernetes-dashboard: dial tcp 192.168.90.100:8443: getsockopt: connection refused

[spnzig]

Can you check if the dashboard addon is enabled? And also in some cases, there may be issues in pulling the sandbox image due to certain firewall rules/policies. Maybe you can use a VPN connection and restart minikube.

[ssuarez6]

Well, I actually checked and the addon is enabled, here's the output:

• addon-manager: enabled
• kube-dns: enabled
• ingress: disabled
• registry: disabled
• registry-creds: disabled
• dashboard: enabled
• default-storageclass: enabled
• heapster: disabled

I'm using actually a VPN and doesn't work, neither with nor without it.

[mykaul]

I am seeing a failure with K8S 1.7.3 (and it works with v1.7.2).
[ykaul@ykaul docker-machine-kvm]$ minikube start --kubernetes-version=v1.7.3 --vm-driver=kvm
Starting local Kubernetes v1.7.3 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.
[ykaul@ykaul docker-machine-kvm]$ minikube dashboard
Could not find finalized endpoint being pointed to by kubernetes-dashboard: Error validating service: Error getting service kubernetes-dashboard: services "kubernetes-dashboard" not found
[ykaul@ykaul docker-machine-kvm]$ minikube status
minikube: Running
localkube: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.42.174

[ykaul@ykaul docker-machine-kvm]$ kubectl get pod --all-namespaces
No resources found.

From minikube logs I'm seeing:
Aug 15 10:03:54 minikube localkube[3201]: E0815 10:03:54.757250 3201 ready.go:58] Error performing healthcheck: Get https://localhost:8443/healthz: x509: certificate is valid for kubernetes.default.svc.cluster.local, kubernetes.default.svc, kubernetes.default, kubernetes, not localhost
Aug 15 10:03:54 minikube localkube[3201]: http: TLS handshake error from 127.0.0.1:54866: remote error: tls: bad certificate

On 1.7.2, this is what I'm seeing:
Aug 15 10:08:01 minikube localkube[3020]: I0815 10:08:01.429352 3020 ready.go:30] Performing healthcheck on http://127.0.0.1:8080/healthz
Aug 15 10:08:01 minikube localkube[3020]: I0815 10:08:01.430076 3020 ready.go:42] Got healthcheck response: [+]ping ok

[furuholm]

@mykaul I see this as well on v1.7.3.

[stela]

@mykaul @furuholm Same here, get these errors running minikube logs:

Aug 21 11:38:21 minikube localkube[3697]: I0821 11:38:21.925434    3697 ready.go:32] Performing healthcheck on https://localhost:8443/healthz
Aug 21 11:38:21 minikube localkube[3697]: http: TLS handshake error from 127.0.0.1:35312: remote error: tls: bad certificate
Aug 21 11:38:21 minikube localkube[3697]: E0821 11:38:21.932843    3697 ready.go:58] Error performing healthcheck: Get https://localhost:8443/healthz: x509: certificate is valid for kubernetes.default.svc.cluster.local, kubernetes.default.svc, kubernetes.default, kubernetes, not localhost

Wiped my ~/.minikube and the issue persists.

$ minikube start --memory 8192 --cpus 4 --vm-driver xhyve --xhyve-disk-driver=virtio-blk --kubernetes-version v1.7.3
$ minikube version
minikube version: v0.21.0

Works fine for me using kubernetes 1.7.2, but 1.7.3 doesn't function.

However, while @ssuarez6's log also had failing health-checks, it doesn't contain the same errors about localhost missing in the certificate. Think this might be a separate issue?

[uromahn]

@mykaul @furuholm @stela I see the same issue with minikube v0.21.0 on OS X.

Here is my output:
Starting: $>minikube start --cpus 4 --disk-size 50g --memory 6144 --kubernetes-version "v1.7.3" --vm-driver xhyve
$>kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.4", GitCommit:"793658f2d7ca7f064d2bdf606519f9fe1229c381", GitTreeState:"clean", BuildDate:"2017-08-17T17:03:51Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"dirty", BuildDate:"2017-08-11T10:05:43Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
$> minikube service list
|-----------|------------|--------------|
| NAMESPACE | NAME | URL |
|-----------|------------|--------------|
| default | kubernetes | No node port |
|-----------|------------|--------------|

As can be seen, there is no kube-dns and dashboard available. I suspect that it is missing from the minikube release for k8s 1.7.3 since k8s 1.7.2 works perfectly well.

[r2d4]

I think this is a result of https://github.com/kubernetes/minikube/commit/8837045970acf954c2613cf7f124bed2233dccc6

This works at head, but we'll need to release a new version of minikube to write the kubeconfig properly for secure serving. cc @dlorenc

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close