Cluster-api: Moving cloud provider specific code out of the repo

[kris-nova]

Right now the cluster-api repository has some cloud provider specific code.

• GCP Deployer (link)
• Google Actuator (link)
  
  
  
  • Terraform Actuator (link)
  
  

The issue of removing the cloud specific code from the repository has been brought up a lot. We mentioned it at Kubecon EU 2018, it's frequent in the slack channel, and mentioned on the weekly calls.

We would like to determine if this is something the project would like to do.

Reasons to remove it

• It encourages a "favored" implementation, and creates a support burden for the project
• It might be out of scope for the project, as cloud providers could support these themselves
• It draws a boundary in the software, encouraging us to adhere to a logical separation of concerns

Reasons to keep it

• It makes testing, versioning, and supporting the code base simpler
• It keeps all pieces together for a single concrete example of a working cluster API implementation
• It encourages the cloud provider specific bits to be community driven

Call for action

Can we please call out other concerns and benefits for either case in this thread?

Also can you please cast a vote on the comments below so we can get a feel for what folks think is the best option moving forward?

The migration process (if it happens) would be documented and planned via the project as all of our other work is done. This is just to determine if we should start the planning process or not.

👍 to vote

[kris-nova]

Please keep the code in the repository!

[kris-nova]

Please remove the code from the repository!

[rsdcastro]

We talked a lot about moving it out of the tree, and if we already had more providers in the repo and we started seeing the clear benefits, I'd be totally supportive.

However, I'd ask you to consider:
(a) we don't have providers other than what Google has been working on so far.
(b) we have a lot of work going on in the common code that isn't tested in isolation but with the provider code which is in the most advanced state.
(c) code ownership can be addressed with OWNERs, which is what Huawei is planning on doing (and the same had been discussed with Loodse)

For most contributors, this adds complexity to managing PRs, vendoring code, etc to most of the contributors without clear concrete short-term gains.

I still think that moving provider code out of the tree should happen, but I don't think this is the right time yet.

[medinatiger]

I'm with Rodrigo. I agree with long term goal of have separation for provider code. But we are not there yet.

[spew]

Concur -- it seems like the benefits won't outweigh the negatives at this point. We should wait for the point at which the providers in the same repo starts creating problems that slow development and at that point split things out.

[weargoggles]

Here's another provider https://github.com/kubermatic/machine-controller/ and it's already out of tree.

edit: reverted edit

[kris-nova]

And another: https://github.com/kubicorn/controller

[jessicaochen]

@kris-nova - Can we have an official vote option for what @spew is suggesting?

[kris-nova]

@jessicaochen how is that different than removing the code from the repository?

Above we mention that we would follow a normal planning cycle for the work. It seems like the timing of when the code would be removed is an implementation detail that falls under the remove the code from the repository plan.

Happy to add a third option - but just seems like it's saying the same thing.

[tamalsaha]

Here is another one: https://github.com/pharmer/pharmer/pull/407

[spew]

I think creating an issue, planning, and scheduling the work to move providers out of the repo is a good thing. However, I'd propose that we explicitly do not block commits adding more functionality to those providers in lieu of that work.

[kris-nova]

Completely agree @spew

[jessicaochen]

I think there is confusion on under which option what @spew is suggesting would land under. I believe @kris-nova is saying it falls under "removing the code". However @spew voted for "keeping the code."

I think either calling out a third option or explicitly specifying which option @spew 's suggestion lands under in the option description would help folks vote accurately.

[m1093782566]

I'm with Rodrigo.

[kris-nova]

I think that keeping the code implies we want to advertise that code is and will always be in the repo.

I think removing the code implies we plan to (some day) remove the code and advertise it as such.

To me removing the code tomorrow, or next year - still falls under removing the code.

[m1093782566]

Thanks for clarification, @kris-nova

[weargoggles]

How about adding new providers to the repo?

[rosti]

Not removing the cloud providers code immediately does imply that new providers code should come into the repo for the foreseeable future. However, we still have to make sure that the CP code remains well separated from the rest of the Cluster API.

[tamalsaha]

In most Kubernetes repos, cloud provider specific is moved out to their own repos to make things manageable. GIven this is a new effort, it makes sense to do that sooner than later.

[kris-nova]

Following up on the meeting from today will be closing this issue with a permanent update on the status of cloud providers (actuators) for the cluster API. (ORIGINAL NOTES FROM CALL)

New repositories for each cloud provider

Per a discussion with the leads of project (@krousey, @roberthbailey, @kris-nova, and @thockin) we feel that the best solution for cluster API controller implementations (actuators) is in independent repositories specific to each cloud.

Each repository should have more than 1 owner spanning multiple companies/interests.

We move the cloud providers of this repository as they are needed, and each new repository will need to be independently maintained.

For information on how to setup a new repository contact the administrators of the cluster-api working group.

[roberthbailey]

So that I don't lose the link, one option would be to move the GCP code to https://github.com/kubernetes/cloud-provider-gcp (and cloud-provider-foo repos for other code) rather than creating new repositories for just the machine controllers.

On the other hand, having a single build / repository is also an attractive model.

It makes sense for each provider to decide how to combine or split up their code as the see fit.