Kubeadm: Graduate kubelet config to beta

[fabriziopandini]

FEATURE REQUEST

Kubelet config was introduced in kubeadm init as alpha feature behind feature gates, but it is planned to graduate it as soon as possible.

It should be necessary to introduce corresponding phases as well before graduation.

[luxas]

/assign @xiangpengzhao
this is a goal for v1.10, but the PR can be sent ASAP. It just can't merge before the v1.9 freeze lifts.

[xiangpengzhao]

TODO list:

• [x] "It should be necessary to introduce corresponding phases as well before graduation." There is already a kubelet phase. Anything else should do here?
  https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/cmd/init.go#L341-L347

if err := kubeletphase.WriteInitKubeletConfigToDiskOnMaster(i.cfg)

• [ ] switch feature gate from alpha to beta and default to true. UPDATE: https://github.com/kubernetes/kubernetes/pull/60412 for 1.11
• [x] load/store initial configuration from/to InitConfigFile instead of InitConfigDir (https://github.com/kubernetes/kubernetes/pull/55718)
  UPDATE: this bullet is invalid now. #53833 uses flag --config to load its initial configuration.
• [x] import pkg/kubelet/apis/kubelet.config.k8s.io/v1beta1 instead of pkg/kubelet/apis/kubeletconfig/v1alpha1 (already done in https://github.com/kubernetes/kubernetes/pull/53833)
• [x] use configmap key kubelet.config.k8s.io instead of kubelet (introduced by https://github.com/kubernetes/kubernetes/pull/53833)

    // KubeletBaseConfigurationConfigMapKey specifies in what ConfigMap key the initial remote configuration of kubelet should be stored
    // TODO: Use the constant ("kubelet.config.k8s.io") defined in pkg/kubelet/kubeletconfig/util/keys/keys.go
    // after https://github.com/kubernetes/kubernetes/pull/53833 being merged.
    KubeletBaseConfigurationConfigMapKey = "kubelet"

UPDATE:

53833 still uses kubelet as the configmap key: https://github.com/mtaufen/kubernetes/blob/9ebaf5e7d27451ef525b3373c46db29221af9cc0/pkg/kubelet/kubeletconfig/checkpoint/configmap.go#L30

The 3, 4, 5 should wait for the associated PR merged.

Anything else am I missing or anything incorrect from above list? @luxas @fabriziopandini

[fabriziopandini]

@xiangpengzhao, with "introduce corresponding phases" I was thinking at kubeadm alpha phases, where we should provide a set of subcommands that allow user to invoke atomically all the steps executed during kubeadm init.

@xiangpengzhao, @luxas, if you agree I will rename this issue into "Graduate kubelet config to beta“

[xiangpengzhao]

@fabriziopandini thanks for clarifying !
+1 from me on renaming the issue title.

[luxas]

Anything else should do here?

@xiangpengzhao it should be exposed as commands in, for example:

• kubeadm phase kubelet config init write
• kubeadm phase kubelet config dynamic enable-for-node
• kubeadm phase kubelet config dynamic upload

[xiangpengzhao]

phase added in https://github.com/kubernetes/kubernetes/pull/57224. The graduation should wait for the associated PRs merged.

[luxas]

FWIW; we need a phase / something command that points a kubelet to a new ConfigMap as well, for the kubelet upgrade scenario.

e.g. I've just upgraded the kubelet (which is associated with the v1.9 configmap) to v1.10, I now need to repoint it to the v1.10 config... maybe we can just document to use kubeadm phase kubelet config dynamic enable-for-node and make an alias for it as well?

[fabriziopandini]

@luxas please check discussion on #57244
currently enable-dynamic-config is the only phase subcommand that implements a step of the kubeadm join sequence (everything else is related to kubeadm init).

IMO we should move this somewhere else, and considering the use case, I suggest under kubeadm upgrade. What about kubeadm upgrade dynamic-kubelet-config --node-name=myNode?

[xiangpengzhao]

I was thinking phase preflight node to be a step of kubeadm join (isn't it?) when I implemented enable-dynamic-config. So I also made enable-dynamic-config a step of kubeadm join.

Agree to document to make them more clear.

[luxas]

I was thinking phase preflight node to be a step of kubeadm join (isn't it?)

It is, yes

So I also made enable-dynamic-config a step of kubeadm join.

:+1:

kubeadm upgrade dynamic-kubelet-config --node-name=myNode?

Something like that probably makes sense... What do people think would be the most productive naming?

kubeadm upgrade kubelet-config [myNode]
kubeadm upgrade node config [myNode]
...etc?

[xiangpengzhao]

@timothysc wrote in https://github.com/kubernetes/kubernetes/issues/53084#issuecomment-367517086 (Graduate the kubeletconfig API to beta)

@ kubernetes/sig-cluster-lifecycle-misc (we should move to enable asap).

I updated the TODO list https://github.com/kubernetes/kubeadm/issues/571#issuecomment-347408384 as per the final implementation of https://github.com/kubernetes/kubernetes/pull/53833. The only bullet in the TODO list is switch feature gate from alpha to beta and default to true. @timothysc are there anything else you were thinking of in your comment?