Kubebuilder: How to generate a controller for two separate CRDs using kubebuilder

[angrosz]

Hi,

kubebuilder book describes how to generate a controller for a single CRD. My task is to create a controller which has to watch objects for 2 separate CRDs and implement custom logic for them. Is it possible to generate a controller for two separate CRDs using kubebuilder commands?

Your assistance in this matter would be greatly appreciated.

Best Regards,
Ania

[droot]

Best practice is to have one controller reconciling objects of one type. So a controller can watch objects of a type, generated objects for that type (or other arbitrary object) as long as all of these changes results in reconciliation of one type. The kubebuilder book explain that using containerset example where you can watch pod type (owned by containerset) and containerset types and you reconcile containerset type.

Will need more info about your use-case here to determine if the two CRDs are related or not. If they are completely independent, then having separate controller for each will be recommended way to go. Would be great if you can share more info about the use-case.

[alegacy]

We have a similar use-case. The application we are implementing is composed of a handful distinct resource types (CRD). They are all related in the sense that, together, they form a single larger system, but only some of them have a strong relational dependency. It is not practical to model them as a single CRD due to the number of attributes and the desire to be able to change and manage smaller parts of the system more independently.

We would prefer to use a tool such as kubebuilder to automatically generate the CRD yaml and validation schema from the *_types.go file rather than hand manage both the CRD and the types.go since it is mostly all duplicated effort. Writing the manager and the controller(s) by hand is something that we are willing to do if necessary but having the framework scaffolded is certainly nice.

Since all of our resources are related we would rather have a single manager process to oversee all things related to our custom resources. We are not opposed to using separate controller objects, if that is architecturally better, but there is a strong desire to have them all exist within the same manager process.

I have found that I can simply add new *_types.go files and re-run "make manifests" and new CRDs are generated. For example, foo_types.go was automatically created when I ran the "kubebuilder create api ... --Kind Foo" and when I can create a bar_types.go and re-run "make manifests" I get the desired result from a CRD point of view but no sample yaml file is generated for this new CRD Type in the "samples" directory.

Since I am new to kubernetes, I don't yet know if this is sufficient or if there is a hidden piece missing that I have not yet discovered. I am also not sure if simply replicating the "controller/foo" directory to make a "controller/bar" directory (with renamed files and content, of course) and then referencing that from within the manager main() is going to provide a successful strategy or whether it is going to interfere with other parts of kubebuilder or any assumptions it may have made about the previously generated code.

I guess what we are looking for is a "kubebuilder create api " command could take multiple "Kind" arguments and scaffold all required files for a multi controller manager process with accompanying CRDs, samples, and test files. For example, "kubebuilder create api --group animals --version v1beta1 --kind Cat --kind Dog --kind Rabbit"

If there is a better approach I would be interested in hearing about it.

[alegacy]

To follow up on my own question... Looks like I could just repeat the "kubebuilder create api" command multiple times.

kubebuilder create api --group animals --version v1beta1 --Kind Cat
kubebuilder create api --group animals --version v1beta1 --Kind Dog
kubebuilder create api --group animals --version v1beta1 --Kind Rabbit
edit *_types.go files
make manifests

[angrosz]

I did exactly the same as @alegacy did.

[benjaminhuo]

To follow up on my own question... Looks like I could just repeat the "kubebuilder create api" command multiple times.

kubebuilder create api --group animals --version v1beta1 --Kind Cat
kubebuilder create api --group animals --version v1beta1 --Kind Dog
kubebuilder create api --group animals --version v1beta1 --Kind Rabbit
edit *_types.go files
make manifests

Thanks for the details @alegacy .
So this way, you can have 3 CRDs and 3 controllers for each CRD and all these 3 controllers could be managed by one single manager in main?

Thanks
Ben

[alegacy]

@benjaminhuo Yes, exactly

[benjaminhuo]

@alegacy Thanks.
So you still have to use one controller to reconcile one CRD, and it's not possible to reconcile multiple CRDs in one controller?

[alegacy]

@benjaminhuo yes, a controller can only handle a single CRD.

[angrosz]

@alegacy
I have 4 CRDs and 4 controllers for each CRD and all these 4 controllers are managed by one single manager in main using _SetupWithManager_ function. Now I have to change the names of all 4 CRDs and I need to have a new version of CONTROLLER that manages both sets of custom resources: the old ones and the new ones. It is important to stress that 4 new CRDs will do exactly the same what the existing ones are doing now. This is a temporary solution to give applications that are using existing CRDs to migrate to the new ones. Should I generate 4 new controllers to be managed with 4 existing ones by one manager in main? Or maybe the other solution exists that doesn't require to add 4 new controllers. I'll appreciate your help.
Best Regards,
Ania

[angrosz]

@droot
I have 4 CRDs and 4 controllers for each CRD and all these 4 controllers are managed by one single manager in main using SetupWithManager function. Now I have to change the names of all 4 CRDs and I need to have a new version of CONTROLLER that manages both sets of custom resources: the old ones and the new ones. It is important to stress that 4 new CRDs will do exactly the same what the existing ones are doing now. This is a temporary solution to give applications that are using existing CRDs to migrate to the new ones. Should I generate 4 new controllers to be managed with 4 existing ones by one manager in main? Or maybe the other solution exists that doesn't require to add 4 new controllers. I'll appreciate your help.
Best Regards,
Ania

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.