Cluster-api: Make MachineSet extensible in a similar way that Clusters/Machines are

[joejulian]

/kind feature

Describe the solution you'd like
Add ProviderSpec to MachineSet

Anything else you would like to add:
In DC/OS we have a tool (MKE) that manages the lifecycle of etcd, kubernetes master components, and nodes on mesos agents. To utilize that capability with a cluster-api implementation, we only want to specify a Cluster and MachineSet and let MKE handle the rest. For that to work, we need to add our own controller logic to the MachineSet.

In our use case, the MachineSet controller would still create Machines, but the Machine controller won't do anything but update the resource state.

[detiber]

This would also enable providers to more easily build around cloud provided auto scaling groups as well.

[davidewatson]

This would also be one way to support an EKS variant for cluster-api-provider-aws. Since there is work in progress to allow the cluster autoscaler to target MachineDeployments/MachineSets (see https://github.com/kubernetes/autoscaler/pull/1481) it would be important to document the semantics of the controllers if different implementations are to interoperate.

When I last looked at this, writting an actuator interface for MachineDeployments looked to be relatively straightford (it basically paralleled the interface for the Cluster actuator). An interface for MachineSets however was not as obvious to me since it attempts to manage specific machines.

If we can get together a design I'd be interested in helping out.

/cc @alvaroaleman

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[vincepri]

/remove-lifecycle stale

[vincepri]

/area api

[timothysc]

@detiber - could you link this with more context.

[detiber]

Azure provider issue: https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/113
/cc @juan-lee @alexeldeib

[alexeldeib]

@cecilerobertmichon

[ncdc]

We are essentially out of time for this to make v1alpha2
/milestone Next

[ncdc]

@samba could you please 👀 and let us know if this is still something your team is interested in. If so, could you please provide more details that line up with the v1alpha2 state of the API?

@CecileRobertMichon is this issue relevant to you? Is it the same thing as the MachinePool proposal?

I'm inclined to close this and have folks open up new issues that are more specific and in line with the v1alpha2 state of the world, if that works?

[CecileRobertMichon]

@juan-lee @vincepri @justaugustus

yes I believe the MachinePool proposal would be an alternative to this.

[ncdc]

@CecileRobertMichon thanks. I'll wait a bit to hear back from either @samba or @joejulian before closing.

[samba]

@ncdc We can close this, at least for now. Our platform needs have changed, and we probably wouldn’t approach this under v1alpha2 (and later) the same way we were thinking about it in the v1alpha1 timeframe. We will follow up in another issue if any related architecture is relevant in the future. Thanks for checking!

[vincepri]

/close

[k8s-ci-robot]

@vincepri: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.