Kubeadm: Ability to configure user and cluster name in AdminKubeConfigFile

[shashidharatd]

When we bring up k8s cluster using kubeadm, the cluster is configured with default cluster-name of kubernetes and user name kubernetes-admin in admin.conf kubeconfig file.
When user wants to deploy multiple clusters and manage them from same kubeconfig file by merging the kubeconfigs of multiple deployments, there will be duplicate user-names but different users, which makes it difficult to merge the kubeconfigs.
So we need the ability in kubeadm to customize the cluster and user-names written to admin.conf

/cc @pipejakob @luxas

[shashidharatd]

/cc @fisherxu

[fisherxu]

@shashidharatd Have send a pr ：)

[fisherxu]

Hi, @pipejakob @luxas, As said in this issue, I think we should make cluster-name and user-name configurable in kubeadm , and can you give some suggestions about this? Thanks :)

[fabriziopandini]

My comments on #52470

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[timothysc]

https://github.com/kubernetes/kubernetes/pull/55901 addresses.

[ps-aux]

@timothysc could you please exlain how https://github.com/kubernetes/kubernetes/pull/55901 addresses custom username/cluster name config ?

[timothysc]

@ps-aux Sorry wrong link.
https://github.com/kubernetes/kubernetes/pull/60852 addresses clustername, and has been merged.

https://github.com/kubernetes/kubernetes/pull/52470 - addressed both, but I'm not sold on kubeadm touching multi-user space. IMO this is policy and can/should be done separately.

As a result, I'm closing this issue.

If folks have a compelling argument on user override that requires putting it into kubeadm, please open a new issue that is scoped to just that problem.

[sekrett]

Is there any option for kubeadm init for specifying cluster name? I did not find it. I created a config file but it is experimental. Such an important thing should be easy to configure.

[neolit123]

@sekrett
on this page search for the config field: clusterName. it can be set in the config.

[sekrett]

As I said before I created a config file and it works. I don't like the fact it's apiVersion is alpha.

[neolit123]

this is how k8s features work, alpha -> beta -> GA.
https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#using-a-feature

there is a command called kubeadm config migrate that will help you to update your config from the alpha version to beta, once the beta version is released.

[sekrett]

@neolit123 Nice information, thanks. That page says that using an alpha version may be buggy and should not be used in long living clusters, that's the thing I am concerned about. I dumped a config file with kubeadm config print-default and it is pretty long. Maybe I should minimise it so that it contains only clusterName and hope it will not lead to bugs?

[neolit123]

@sekrett

Nice information, thanks. That page says that using an alpha version may be buggy and should not be used in long living clusters, that's the thing I am concerned about.

moving the config away from alpha is a slow process. we expect the config to be beta in 1.12.

I dumped a config file with kubeadm config print-default and it is pretty long. Maybe I should minimise it so that it contains only clusterName and hope it will not lead to bugs?

you can extract only the parts that you need from print-default.

[sekrett]

Got it, thanks!

[caboteria]

Hi! I just bumped into this issue when I tried to do the same thing as OP, i.e., build two clusters with kubeadm then configure kubectl to control both of them. When you provide both config files to kubectl (for example, by putting both in $KUBECONFIG) it can distinguish between both clusters because they can have different names, but because both admin users have the same name kubectl picks only one and uses that so it can access only one of the two clusters.

It would be very useful to be able to specify the admin user name when the cluster is built since then kubectl could control multiple kubeadm-created clusters.

@timothysc could you please elaborate on your comment "I'm not sold on kubeadm touching multi-user space. IMO this is policy and can/should be done separately."? How can I do this separately? It seems like I get a user named kubernetes-admin with no way to change that, which feels like a policy decision to me.

I'd be willing to look at making a new PR with just the user name part of kubernetes/kubernetes#52470 but don't want to waste my time if it will get shot down.

Thanks!

[neolit123]

@caboteria

hi, in kubeadm 1.20 (to be released) we extended the command kubeadm alpha kubeconfig to accept the ClusterConfiguration object:
https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2

this allows you to generate kubeconfig files for different clusters / users / orgs.

we cannot backport this extension to older releases, but technically you could use the 1.20 binary to generate kubeconfig for older clusters.

[caboteria]

hi @neolit123 , thanks for the info! That looks like an improvement but I don't see anything about setting the admin user name. I'm still learning my way around k8s, am I missing something? It looks like I'll still be unable to init a cluster with an admin user named anything but kubernetes-admin.

Thanks again!

[neolit123]

the new command allows --client-name:
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/cmd/alpha/kubeconfig.go#L97

so you could do this command kubeadm alpha kubeconfig user --client-name=some-admin --org=system:masters --config=some-file-with-ClusterConfiguration

ClusterConfiguration can be used for the:

• controlPlaneEndpoint - apiserver address in the kubeconfig
• clusterName - name of the cluster

[caboteria]

Thank you @neolit123, that works!

Back to https://github.com/kubernetes/kubeadm/issues/416#issuecomment-719798555 :

I'd be willing to look at making a new PR with just the user name part of kubernetes/kubernetes#52470 but don't want to waste my time if it will get shot down.

I think it would be a better user experience to specify the default admin user name (like we can specify the cluster name) instead of having to create a second user.

Another potential approach would be to have the default admin name be based on the cluster name, e.g., clusterName+"-admin". Then if the client changed the cluster name the admin name would track that change. WDYT?

[neolit123]

this idea was rejected before, but if you feel strongly about it, please log a new issue with a detailed description and link to this old ticket. it can be discussed for v1beta3. thanks.