Cluster-api: [cabpk] Show the actual kubeadm configuration applied to a machine

[dlipovetsky]

/kind feature

Describe the solution you'd like
The typical kubeadm configuration generated by CABPK is minimal, and kubeadm applies many defaults, the values of which are not observable in any CAPI objects. Today, my options are to inspect /tmp/kubeadm.yaml on the machine, or fetch the kubeadm configuration from the cluster's control plane.

Anything else you would like to add:
Being able to observe the actual configuration applied to the machine will improve the debugging experience.

[chuckha]

does /tmp/kubeadm.yaml provide all the data you want or do you want the full configuration with all the defaults applied?

[detiber]

/tmp/kubeadm.yaml should be pretty much identical to the bootstrap data (with the exception of the jinja template expanded in the case of CAPA)

[dlipovetsky]

Controllers have access to the workload cluster API. If the information is available in some object (e.g. a configmap) on the workload cluster API, then presumably a controller can pull it and update the KubeadmConfig status.

With this in mind:
(a) Can we get the info from objects kubeadm already posts? If not, then:
(b) Can we add a step to the bootsrap script that creates such an object? (e.g. a configmap with the output of kubeadm --dry-run, if that's the right way to get the actual configuration applied)

[chuckha]

You could do that yourself today with PostKubeadmCmds!

[fabriziopandini]

@dlipovetsky kubeadm already stores the Cluster Configuration in the kubeadm-config ConfigMap, is that what are you looking for?

[dlipovetsky]

@dlipovetsky kubeadm already stores the Cluster Configuration in the kubeadm-config ConfigMap, is that what are you looking for?

Thanks for pointing this out. Yes, ClusterConfiguration is good to have. I would also like the _actual_ (i.e. with all default values applied) InitConfiguration/JoinConfiguration used for the Machine.

(I'm about to file a related kubeadm issue)

[detiber]

@dlipovetsky I'm not sure that is actually ever stored anywhere on disk, I believe it's all in-memory for the init/join workflows.

[ashish-amarnath]

Would making kubeadm write the config out as a log message at a very high verbosity level be an acceptable fix for this?

[dlipovetsky]

@dlipovetsky I'm not sure that is actually ever stored anywhere on disk, I believe it's all in-memory for the init/join workflows.

You're right.

But I think the information is useful to have, although we'll have to keep in mind that some values (e.g. the bootstrap token) are sensitive.

[chuckha]

Would making kubeadm write the config out as a log message at a very high verbosity level be an acceptable fix for this?

The problem is that kubeadm doesn't show default values nor does it write default values to disk at any point for us to write to a log.

[chuckha]

@dlipovetsky I'm not sure that is actually ever stored anywhere on disk, I believe it's all in-memory for the init/join workflows.

You're right.

But I think the information is useful to have, although we'll have to keep in mind that some values (e.g. the bootstrap token) are sensitive.

Thankfully the kubeadm authors thought of this. JSONUnmarhsal functions are implemented for sensitive fields! For example: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/apis/kubeadm/v1beta2/bootstraptokenstring.go#L44

edit: what I actually meant was that the fields are omitted from JSON using struct tags: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/apis/kubeadm/v1beta2/bootstraptokenstring.go#L34

[ashish-amarnath]

@dlipovetsky I meant we can make kubeadm write this out as a log message at a high verbosity level.
Like unmarshal the config structs and write them to the log.

[chuckha]

oh I see what you mean...not that I'm aware of but I've been out of the kubeadm codebase for a hot minute

[dlipovetsky]

Would making kubeadm write the config out as a log message at a very high verbosity level be an acceptable fix for this?

My goal is to bubble up the config somewhere, e.g., to a KubeadmConfig object. It's a pain to parse it from a log, but trivial with a file.

[neolit123]

commented here:
https://github.com/kubernetes/kubeadm/issues/1829#issuecomment-540197491

[chuckha]

This is blocked on kubeadm implementing this feature for us to use and won't be coming until at least 1.17+

Until then it will be up to the user to dump as much kubeadm config as they can

/milestone Next

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[vincepri]

/lifecycle frozen

[vincepri]

/area ux
/area bootstrap
/remove-lifecycle frozen
/help
/priority backlog

[k8s-ci-robot]

@vincepri:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/area ux
/area bootstrap
/remove-lifecycle frozen
/help
/priority backlog

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[vincepri]

/lifecycle frozen