Kubeadm: Switch to generating ECDSA keys

[rojkov]

FEATURE REQUEST

To unify the type of certificates used across Kubernetes make kubeadm generate ECDSA keys and certificates instead of RSA ones during cluster initialization and key renewal.

Motivation

As explained in OpenSSL wiki the primary advantage of using Elliptic Curve based cryptography is reduced key size and hence speed.

In addition to that kubelet already generates self-signed server certificates of the ECDSA type and uses ECDSA client certificates when communicating with API servers. It would reduce complexity if the same defaults were used across Kubernetes components.

Goals

• unconditionally generate ECDSA keys upon cluster initialization.

Proposal

Since people feel more comfortable when there is a transition period it makes sense to introduce a configuration option first for the type of keys used in new clusters or the clusters undergoing upgrades.

The selected key type should be defined in InitConfiguration or optionally overridden with a command line option applicable to the init command and to the certs related subcommands.

The function pkiutil.NewPrivateKey() should accept keyType parameter of two possible values ECDSA and RSA. The actual value is passed from InitConfiguration by callers. Currently the function unconditionally generates RSA keys only.

The default value for the option is RSA.

It is OK to have certificates of mixed types in the same certificate chain thus after two or three releases the default value can be switched to ECDSA.

After the transition period is over the option can be removed completely and all new keys are going to be of the ECDSA type.

[fabriziopandini]

@rojkov is it possible to merge this issue with https://github.com/kubernetes/kubeadm/issues/984?

[rojkov]

@fabriziopandini I don't think it's needed as they are different issues: #984 is about kubeadm accepting pre-generated ECDSA certs (without kubeadm) and this issue is about kubeadm generating ECDSA certs itself.

[timothysc]

@rojkov - could you please attach your PRs to this issue and what else needs to get done here?

/cc @kad

[neolit123]

xref https://github.com/kubernetes/kubernetes/pull/76390

[neolit123]

moving to melestone Next.

https://github.com/kubernetes/kubernetes/pull/76390
added support for user generated ECDSA.

this ticket tracks the generation of ECDSA by kubeadm.

After the transition period is over the option can be removed completely and all new keys are going to be of the ECDSA type.

we probably should define when this transition is going to happen.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[anguslees]

ECDSA keys are significantly faster fwiw, particularly on arm32. I could not enable TLS peering certs on my rPi2 cluster and have the health checks (TLS handshakes) complete in less than the etcd hard-coded connect timeout with RSA keys - ECDSA keys work fine.

[fabriziopandini]

@rojkov thanks for the feedback.
@neolit123 @timothysc we should start to consider if/how to make ECDSA became the kubeadm default

[neolit123]

@anguslees

ECDSA keys are significantly faster fwiw, particularly on arm32. I could not enable TLS peering certs on my rPi2 cluster and have the health checks (TLS handshakes) complete in less than the etcd hard-coded connect timeout with RSA keys - ECDSA keys work fine.

i think i've seen related issues before.
the workaround until now was patching the timeouts to be longer.

but isn't 15 seconds a bit too much even for rpi?

people were considering race conditions at some point on ARM...
also the kubeadm kubeconfig logic randomly panic-ed for a certain rpi2 user and that did not make any sense (possibly compiler bugs?).

rpi in general is not supported until we have e2e tests for it.

@fabriziopandini

we should start to consider if/how to make ECDSA became the kubeadm default

+1 if @rojkov can get back to this work.

[rojkov]

I might get back to this. Will update tomorrow or on this Wednesday.

[rojkov]

I need to finalize one my PR to Envoy's main repo and one more PR to the OpenSSL extension to Envoy first. They take all my time budget allowed for open source contributions. Then I can continue with ECDSA keys.

[neolit123]

thanks @rojkov

[rojkov]

Submitted https://github.com/kubernetes/kubernetes/pull/86953

[neolit123]

@rojkov
during the kubeadm meeting today we discussed the alternative option to use feature gates instead of adding this as part of the API, but we didn't make a clear decision.

i'm personally in favor of adding this as part of the API, but it means it has to wait until we have enough new features for v1beta3. we might not be able to release v1beta3 this cycle because kubeadm is moving out of k/k and that's where most of the effort will be.

if you want to add this as part of kubeadm ASAP, then it has to be a future gate and only work on init. my problem with that is that the new cluster certs will be stuck to ECDSA.

ideally i wish we support renew paths that support transitioning old certs to ECDSA from RSA, or the other way around for people to opt-out of the feature gate.

[fabriziopandini]

I'm +1 for a feature gate now with some documentation about the scope of the current implementation (e.g only new clusters, no opt out, yes renewal)

In parallel, a KEP will help in figuring out all the user-stories (e.g opt-in/opt-out, mixed old/new certs)

[rojkov]

@neolit123 @fabriziopandini Thanks for your fast reply!

I'll rework the PR to have an init only feature gate for ECDSA and write a KEP for the rest of possible scenarios (or better the ones I can come up with).

[neolit123]

@rojkov thank you!

[stealthybox]

Thanks for working on this.

What's the rough order of operations to manually move a cluster from RSA to ECDSA.

I can guess that there are dependencies on certain flags in components and the communication between those components. (kube-apiserver / kubelet / etcd / ....)

Is downtime required in some or all upgrade techniques? (multiple controlplane nodes?)

If we can document this with the gated feature, it will serve users and maybe point us toward a more automated behavior.

[rojkov]

@stealthybox It's safe to have keys of different types in a single certificate bundle or chain. I don't expect problems if to add the ECDSA feature gate to ClusterConfiguration of an existing cluster and then renew the keys. But it makes sense to test first :)