Kubebuilder: Proposal: Create APIs and ClientSets in a separate repository

[kdada]

kubebuilder creates projects with APIs and clientset/informer/lister. This behavior causes developers have to maintain APIs in different repos. When users needs to interact with these APIs, they also need to import lots of clients. It's not easy to use!

In a company or an org, developers used to maintain APIs in one place. So I think kubebuilder can help developers to create a project for APIs and clientset (like client-go).

The works of kubebuilder:

1. kubebuiler api init {project-path} creates a new APIs project with:
   
   
   
   • client-gen
   
   
   • informer-gen
   
   
   • lister-gen
   
   
   • conversion-gen
   
   
   • deepcopy-gen
   
   
   • defaulter-gen
   
   
2. kubebuiler api create adds a new API to APIs project
3. kubebuilder init --api={project-path} can initialize a project with external API project.
4. kubebuiler create --api={project-path} can create new controller with external API project

For more convience, we can implement a uniform clientset which inherit client-go(which I'm using now):

import kubernetes "k8s.io/client-go/kubernetes"

type Interface interface {
    kubernetes.Interface
    MyapiV1beta1() myapiv1beta1.MyapiV1beta1Interface
    // Deprecated: please explicitly pick a version if possible.
    Myapi() myapiv1beta1.MyapiV1beta1Interface
}

// Clientset contains the clients for groups. Each group has exactly one
// version included in a Clientset.
type Clientset struct {
    *kubernetes.Clientset
    myapiV1beta1   *myapiv1beta1.MyapiV1beta1Client
}

Then users can import this API project and only need to create ONE client for kubernetes.

[droot]

@kdada Kubebuilder project uses controller-runtime and that has a dynamic client (https://github.com/kubernetes-sigs/controller-runtime/tree/master/pkg/client) which can be used for different types. All you need to do is add an external type to the scheme. So you already have that ONE client :). You don't need to use generated clientset (which kubebuilder doesn't generate by default anyways).

[kdada]

@droot controller-runtime provides a dynamic client. But it is only for one controller project. When users want to CRUDs these APIs, they need to import them from many projects. This is not a tolerable solution in enterprises. So I think it's better to create a unique APIs repo and build a union client for users.

[ScorpioCPH]

+1 for this proposal, here is our scenario:

• we have a CRD and controller which generated from kubebuiler.
• we want to add some admission control for this CRDs in kube-apiserver's admission plugins, so we need a informer to List this CRD, but kubebuiler don't export any client pkg for this CRD.

[ScorpioCPH]

/cc @droot

[DirectXMan12]

@ScorpioCPH @kdada if this is still an issue for you, can you elaborate a bit on why just using the controller-runtime informers and client is insufficient? Those can be imported from any project. We're unlikely to start generating clients and informers (for reasons that we've laid out in our design overview), but I'd like to understand what the issue is here.

[kdada]

@DirectXMan12 Maintaining APIs in multiple projects is not convenient. We want to keep our APIs in one project.Then we can generate clientset for ops tools and those projects which does not import controller-runtime. Also we can generate clients for other programing languages. If a project created by kubebuilder can import API from another project and auto-generate config files (CRD, RBAC, Deployment files), it'd be more preferred.

[kdada]

How I achieve it now:

manifests:
    rm -rf pkg/apis/my-api-group
    cp -r vendor/my-org/clientset/pkg/apis/my-api-group ./pkg/apis/
    go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
    rm -rf  pkg/apis/my-api-group

[DirectXMan12]

If a project created by kubebuilder can import API from another project and auto-generate config files (CRD, RBAC, Deployment files), it'd be more preferred.

For CRD and RBAC, controller-tools v0.2.0 can already do this today. For example, $ controller-gen crd paths=my-org/clientset/pkg/apis/my-apigroup output:crd:artifacts:config=config/crd/bases. No copying around directories needed. You'll probably have to tweak your makefile to include that instead of the default instruction, but that's how you'd do it.

[DirectXMan12]

Is that sufficent ^ ?

[kdada]

@DirectXMan12 It's enough. Thanks for your comments.