Minikube: --extra-config still fails to start with v0.23

[Morriz]

After having read (closed) issue #2064, I want to reopen the same issue again here, as this still fails for me if I start with:

minikube start \
    --kubernetes-version v1.8.1 \
    --bootstrapper kubeadm \
    --v 10 \
    --logtostderr \
    --extra-config=apiserver.Admission.PluginNames="Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,GenericAdmissionWebhook,ResourceQuota,PodPreset"
    --registry-mirror=http://localhost:6000

The boot script hangs and the api-server does not start correctly. I see :8443: getsockopt: connection refused errors in the journal.

If I leave the --extra-config part out it boots correctly, but I can't use my flags ;p

I'm on OSX 10.12.6

[r2d4]

The extra config flag works differently for the kubeadm bootstrapper.

https://github.com/kubernetes/minikube/blob/master/docs/configuring_kubernetes.md#kubeadm-bootstrapper

[Morriz]

Ah, but then I still don't see how I should format the --extra-config=apiserver.Admission.PluginNames= part above. Seems the same to me. Would you show me?

[r2d4]

@Morriz It should take the form of whatever flag you would pass into the apiserver. In this case

--extra-config=apiserver.admission-control="Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,GenericAdmissionWebhook,ResourceQuota,PodPreset"

[Morriz]

Thanks. Is there any straightforward documentation about this? I couldn't figure it out from that link only...

[r2d4]

The documentation that I linked is the only documentation available. I'll be adding more documentation soon from a presentation I'm putting together about minikube.

This will also be a lot clearer once we deprecate localkube, which has the old behavior for the --extra-config flag. I agree its pretty confusing that the flag has different behavior and accepts different values based on the value of another flag, the bootstrapper flag.

A possibly easier way to think about the configuration:

Localkube uses reflection to set the key value pairs on the structs themselves.
Kubeadm passes along the key-value pairs as flags to each component, the apiserver, kubelet, etc.

[Morriz]

Ok, can I reopen this here? I still get errors with:

minikube start \
    --bootstrapper kubeadm \
    --kubernetes-version v1.8.3 \
    --extra-config=apiserver.admission-control="NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds,GenericAdmissionWebhook,PodPreset" \
    --network-plugin=cni \
    --host-only-cidr 172.17.17.1/24 \
    --extra-config=kubelet.pod-cidr=192.168.0.0/16 \
    --extra-config=proxy.cluster-cidr=192.168.0.0/16 \
    --extra-config=controller-manager.cluster-cidr=192.168.0.0/16 \

the error being:

Moving files into cluster...
E1118 12:36:26.396899   41381 start.go:222] Error updating cluster:  generating kubeadm cfg: generating extra component config for kubeadm: Unknown component map[apiserver:apiServerExtraArgs controller-manager:controllerManagerExtraArgs scheduler:schedulerExtraArgs kubelet:].  Valid components and kubeadm config are map[apiserver:apiServerExtraArgs controller-manager:controllerManagerExtraArgs scheduler:schedulerExtraArgs kubelet:]

[r2d4]

Proxy isn't configurable with the kubeadm bootstrapper yet.

Valid components and kubeadm config are map[apiserver:apiServerExtraArgs controller-manager:controllerManagerExtraArgs scheduler:schedulerExtraArgs kubelet:

In the meantime, its not a great solution but you can try to modify the proxy args after it starts up and redeploy.

[Morriz]

hmmm, I am spending a lot of time trying to get calico to work on minikube...maybe you can point me to some info? I can't seem to get it to work with localkube neither :|

[ctaggart]

I was also trying to get calico to work on minikube and ran into the same error reported by @Morriz. I was trying to run:

minikube start -v=10 \
  --vm-driver=hyperkit \
  --network-plugin=cni \
  --host-only-cidr=172.17.17.1/24 \
  --extra-config=kubelet.network-plugin=cni \
  --extra-config=kubelet.PodCIDR=192.168.0.0/16 \
  --extra-config=proxy.ClusterCIDR=192.168.0.0/16 \
  --extra-config=controller-manager.ClusterCIDR=192.168.0.0/16

Any idea how to get past this error so that I can get Clalico running on Minikube?

minikube version
minikube version: v0.27.0

[monte-hayward]

@r2d4 is proxy config support through kubeadm scheduled to be implemented?

Proxy isn't configurable with the kubeadm bootstrapper yet.

In the meantime, here's the only workaround I've found.
Virtualbox CPU usage goes from 140% to ~34%.
It uses deprecated localkube, so proxy config support through kubeadm will be needed when localkube dies.

minikube start --bootstrapper localkube --v=3 --vm-driver=virtualbox --cpus=4 \
--memory=4096 --network-plugin=cni \
--extra-config=proxy.IPTables.SyncPeriod.Duration="5000000000" \
--extra-config=proxy.IPTables.MinSyncPeriod.Duration="3000000000"
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Check network to re-create if needed...
Waiting for an IP...
Waiting for SSH to be available...
Detecting the provisioner...
Setting Docker configuration on the remote daemon...
Getting VM IP address...
WARNING: The localkube bootstrapper is now deprecated and support for it
will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which
is intended to replace the localkube bootstrapper. To disable this message, run
[minikube config set ShowBootstrapperDeprecationNotification false]
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.
Loading cached images from config file.

@Morriz did you find an interim solution?