Minikube: kubeadm init: failure loading apiserver certificate: the certificate is not valid yet

[kyledakid]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Please provide the following details:

Environment:

Minikube version (use minikube version): minikube version: v0.26.0

• OS (e.g. from /etc/os-release): MacOS Siera 10.12
• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): virtualbox
• ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): "Boot2DockerURL": "file:///Users/122462/.minikube/cache/iso/minikube-v0.26.0.iso",
• Install tools: brew cask install minikube
• Others:
  The above can be generated in one go with the following commands (can be copied and pasted directly into your terminal):

minikube version
echo "";
echo "OS:";
cat /etc/os-release
echo "";
echo "VM driver": 
grep DriverName ~/.minikube/machines/minikube/config.json
echo "";
echo "ISO version";
grep -i ISO ~/.minikube/machines/minikube/config.json

What happened:
Executed minikube start
What you expected to happen:
minikube successfully starts
How to reproduce it (as minimally and precisely as possible):
Execute minikube start
Output of minikube logs (if applicable): (not actually minikube log, I got this when executed minikube start)

E0411 16:57:04.402812    4049 start.go:276] Error starting cluster:  kubeadm init error sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap  running command: : running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap
 output: [init] Using Kubernetes version: v1.10.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[certificates] Using the existing ca certificate and key.
    [WARNING Swap]: running with swap on is not supported. Please disable swap
Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.
failure loading apiserver certificate: the certificate is not valid yet
: running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap
.: Process exited with status 1

Anything else do we need to know: VirtualBox 5.2.8

[kylehodgetts]

I've got this here too.

I even tried minikube delete and minikube start and from fresh this issue still raises its head.

After running minikube logs, this line caught my eye.
minikube kubelet[3329]: error: failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory

I'm not sure if this has significance or not

edit: Boom. Here we go:

output: Error: unknown flag: --ignore-preflight-errors
Usage:
  kubeadm init [flags]

[mrballcb]

I'm getting the same kubeadm error (unknown flag --ignore-preflight-errors) when trying to start kube 1.8.10 in minikube. I can start kube 1.8.0 with minikube 0.25.2, the highest 1.8.x that it supports. But I cannot start kube 1.8.10 with minikube 0.26.0 because the arg to kubeadm is not supported.

When you look at the kubeadm command minikube is trying to run, in Jan 2018 it was changed from “--skip-preflight-checks” to a bunch of “--ignore-preflight-errors" args. The variable kubeadmInitTemplate in pkg/minikube/bootstrapper/kubeadm/templates.go is hard coded to use it regardless of what kube version you tell it to use. That appears to be a new 1.9.x and 1.10.x arg. The kubeadm downloaded for 1.8.x does not support the new arg, hence the error. The kubeadm downloaded for 1.9.x does, but something chokes when starting these things up, and it’s not clear to me on what.

[leanne73]

+1

I also got the same unknown flag: --ignore-preflight-errors error when trying to start a k8s v1.8 cluster with minikube 0.26.

I was able to work around it temporarily by downgrading to minikube 0.25.2, thanks for the tip @mrballcb!

[afbjorklund]

Needs to be conditionalized for kubeadm 1.8, the old arg was deprecated in 1.9 that was why. #2403

Flag --skip-preflight-checks has been deprecated, it is now equivalent to --ignore-preflight-errors=all

[afbjorklund]

Obviously storing the kubeadm log and testing the releases with 1.5.x, 1.6.x, 1.7.x, 1.8.x, 1.9.x, 1.10.x would also be good (although I guess at least some of those must be truly deprecated by now... ?)

Localkube says it supports 1.3.x and up, but then again it has been deprecated itself - so what does it know. If I read the Kubernetes docs correctly, they say that 1.7.x and earlier are now end-of-life ? But not 1.8

https://github.com/kubernetes/community/blob/master/contributors/design-proposals/release/versioning.md

[kyledakid]

Minikube v0.26 works fine on Ubuntu 16.04. This is not fair :(

[afbjorklund]

@mrballcb : The choking thing on 1.9 is probably #2716 ? (workaround for CRI, that got fixed in 1.10)

[KowalczykBartek]

as workaround - I started minikube from this release https://github.com/kubernetes/minikube/releases/tag/v0.25.1
At least works on hight sierra 10.13.3

[mgdevstack]

Minikube 0.26.0 cluster initialization still failing. My dependency is on k8s-v1.10 cluster.

How soon can we expect new version release with few important fixes like #2721

Is anyone able to run Kubernetes cluster 1.10 with Minikube 0.26.0 ?

[javajon]

Since Minikube version 0.26.0 I am getting this error too with "minikube start" on Windows 10 + VirtualBox 5.2.8 (latest). I have been sticking with 0.25.2.

Related to this report I captured my details in issue #2818.

[MaLub]

I'll get same issue here:
output: [init] Using Kubernetes version: v1.10.0 [init] Using Authorization modes: [Node RBAC] [preflight] Running pre-flight checks. [certificates] Using the existing ca certificate and key. [WARNING Swap]: running with swap on is not supported. Please disable swap Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version. failure loading apiserver certificate: the certificate is not valid yet : running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI .: Process exited with status 1

Go back to 0.25.1 works for me.

Mac OS 1.12.6

[edwardzxw]

+1
Met with the exactly same problem on CentOS 7 + VirtualBox 5.1, downgrade to 0.25.2 fixed this problem.

[raghur]

Still the same error on 0.27, Win 8.1 and Vbox 5.1; Going back to 0.25.2 as that's the last known good version for me.

Why is minikube upgrade always so error prone :(.. Thankfully, its easy to create a new profile and test with that rather than muck with an existing working installation.

E0608 09:53:15.150680    3544 start.go:276] Error starting cluster:  kubeadm init error sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI  running command: : running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI
 output: [init] Using Kubernetes version: v1.10.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[certificates] Using the existing ca certificate and key.
                                                                                                    [WARNING Swap]: running with swap on is not supported. Please disable swap
Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.
failure loading apiserver certificate: the certificate is not valid yet
                                                          : running command: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI
.: Process exited with status 1

[Rouche]

I dont know if its related...
Everything works fine with 0.28 and kubernetes 1.10.0 Windows 10 Pro + Virtualbox.

But when i try to enable Audit : /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory

Start command:
minikube start --extra-config=apiserver.Authorization.Mode=RBAC --extra-config=apiserver.Audit.LogOptions.Path=/var/log/audit.log --extra-config=apiserver.Audit.PolicyFile=/etc/kubernetes/addons/audit-policy.yaml

[tavyy]

The same problem for me.

I'm running minikube 0.28 and kubectl 1.10.3 on MacOS High Sierra 10.13.5

I used both commands:

minikube start --extra-config=apiserver.Authorization.Mode=RBAC --extra-config=apiserver.Audit.LogOptions.Path=/var/log/audit.log --extra-config=apiserver.Audit.PolicyFile=/etc/kubernetes/addons/audit-policy.yaml

and

minikube start --extra-config=apiserver.authorization-mode=RBAC --extra-config=apiserver.audit-log-path=/var/log/audit.log --extra-config=apiserver.audit-policy-file=/etc/kubernetes/addons/audit-policy.yaml

This is the log file:

Jun 25 19:24:08 minikube kubelet[2616]: F0625 19:24:08.210031 2616 server.go:233] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Jun 25 19:24:08 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jun 25 19:24:08 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.