Minikube: Support for Kubernetes 1.12

[robrich]

Is this a BUG REPORT or FEATURE REQUEST? (choose one): FEATURE REQUEST

K8s v. 1.11 and 1.12 in minikube please?

Please provide the following details:

Environment:

Minikube version (use minikube version): 0.28.2

• OS (e.g. from /etc/os-release): Windows 10
• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): hyperv
• ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): v0.28.1
• Install tools:
• Others:

[geerlingguy]

I'm also wondering—is there any way to get a 1.10 version later than 1.10.0? It has a fairly annoying bug (https://github.com/kubernetes/kubernetes/issues/54870) that causes my minikube cluster to kill itself when testing lots of jobs that fail.

[geerlingguy]

See also: https://github.com/kubernetes/minikube/issues/2988

It looks like 1.11 at least is already there, it's just kinda hidden?

[geerlingguy]

Examples:

minikube start --kubernetes-version v1.10.7
minikube start --kubernetes-version v1.11.3

[tamalsaha]

I tried the same approach for 1.12.0-rc.2 and it did not work.

$ minikube delete; minikube start --kubernetes-version=v1.12.0-rc.2
Deleting local Kubernetes cluster...
Machine deleted.
Starting local Kubernetes v1.12.0-rc.2 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubelet v1.12.0-rc.2
Downloading kubeadm v1.12.0-rc.2
Finished Downloading kubeadm v1.12.0-rc.2
Finished Downloading kubelet v1.12.0-rc.2
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
E0926 15:34:09.552031   16740 start.go:300] Error starting cluster:  kubeadm init error 
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI  &&
sudo /usr/bin/kubeadm alpha phase addon kube-dns
 running command: : running command: 
sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI  &&
sudo /usr/bin/kubeadm alpha phase addon kube-dns

.: Process exited with status 1
================================================================================
An error has occurred. Would you like to opt in to sending anonymized crash
information to minikube to help prevent future errors?
To opt out of these messages, run the command:
    minikube config set WantReportErrorPrompt false
================================================================================
Please enter your response [Y/n]: 

[outcoldman]

I have found several incompatibility issues in minikube with Kubernetes v1.12

1. It uses kubeadm.k8s.io/v1alpha1 for kubeadm. Kubernetes 1.12 does not support version v1alpha1. Only v1alpha2. The recommended version is v1alpha3.

2. This addon is not available /usr/bin/kubeadm alpha phase addon kube-dns, I guess it should use coredns instead

3. The argument cadvisor-port is not available in k8s-1.12

After I have fixed these issues I could bootstrap kube 1.12 with minikube.

I have a patch ready in https://github.com/outcoldman/minikube/tree/k8s-1.12, you can test it with (instructions are for macOS, for other OS change the make target out/minikube-darwin-amd64)

git clone [email protected]:outcoldman/minikube.git
cd minikube
git checkout k8s-1.12
docker run --rm -v "$PWD":/go/src/k8s.io/minikube -w /go/src/k8s.io/minikube -e IN_DOCKER=1 golang:stretch  make out/minikube-darwin-amd64
./out/minikube-darwin-amd64 start --kubernetes-version v1.12.0

[afbjorklund]

We found some v1.11.0 CRI-O issues in #3153, I guess these would also apply to 1.12

[robpblake]

Examples:

minikube start --kubernetes-version v1.10.7
minikube start --kubernetes-version v1.11.3

Building on @geerlingguy's comment. If you're forgetful like me, you can alternatively use

minikube config set kubernetes-version <kubernetes_version>
e.g. 
minikube config set kubernetes-version v1.11.3

Which stores your desired version of Kubernetes more persistently into ~/.minikube/config/config.json and works across multiple restarts of your minikube environment.

[pnpavlov]

hey @robrich, just checked that "minikube start --kubernetes-version=v1.12.0" works with minikube version: v0.30.0. Maybe this issue could be closed?

[psura]

Same here. With minikube v0.30.0, kuvernetes-version v.1.12.0 worked for me and not v1.12.1
For those the latest v1.12.1 works, it could be the mac OS version in the mix as well. My Mac OS Version High Sierra 10.13.6 (17G65) My Colleague with mac OS version El Capitana, the latest minikube v1.12.1 worked fine.

[robrich]

@pnpavlov: Not sure. Maybe we need https://github.com/kubernetes/minikube/pull/3180 and https://github.com/kubernetes/minikube/pull/3262 and https://github.com/kubernetes/minikube/issues/3164#issuecomment-425327164 to land before it's good enough for mortals?

[tstromberg]

CI is in somewhat stable state (<15% failures), so I've finally proposed to make v1.12.3 the default: #3402

[robrich]

Reopen this now that #3402 is unmerged via #3423? Or close in favor of #3386 leading the charge towards v 1.12?