Cluster-api: KCP: ensure machine that is being deleted during upgrades or scale down isn't the etcd leader

[randomvariable]

Detailed Description

For correctness, the control plane controller should ensure etcd leadership is on a machine that isn't about to be deleted. Two optimisations are possible:

• Where there are at least two candidates for deletion, pick the machine which is not etcd leader
• Where there is only one candidate, move the etcd leadership to a different machine.

/kind feature
/help

[k8s-ci-robot]

@randomvariable:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

Detailed Description

For correctness, the control plane controller should ensure etcd leadership is on a machine that isn't about to be deleted. Two optimisations are possible:

• Where there are at least two candidates for deletion, pick the machine which is not etcd leader
• Where there is only one candidate, move the etcd leadership to a different machine.

/kind feature
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[detiber]

I would argue that we should probably always handle this by moving the leadership, otherwise we risk causing conflicts with the failure domain handling.

[vincepri]

In terms of milestone, when do we want to tackle this?

[detiber]

I think we still want to attempt to get this in prior to v0.3.0.
/milestone v0.3.0

[vincepri]

/priority important-soon

[alexander-demichev]

/assign @alexander-demichev

[vincepri]

Are we planning to do this for v0.3.0 or should we punt this to v0.3.x?

[chuckha]

let's leave it in v0.3.0 and if it looks like it will merge today i'll put it in the RC milestone

[michaelgugino]

We need to update the first comment or title to reflect that this is specifically about upgrades.

[vincepri]

/retitle KCP: ensure machine that is being deleted during upgrades isn't the etcd leader

[vincepri]

/milestone v0.3.x