Cluster-api: Deploy cert-manager from clusterctl

[tahsinrahman]

User Story

As a developer I would like to deploy cert-manager from clusterctl for adding webhooks

Detailed Description

We need to deploy cert-manager for provisioning the certificates for the webhook server.
We need to update clusterctl to deploy cert-manager first, wait for cert-manager pods to be ready. Otherwise when applying manifests, we'll get following errors

Internal error occurred: failed calling webhook "webhook.certmanager.k8s.io": the server is currently unable to handle the request

/kind feature

[tahsinrahman]

/assign

[vincepri]

How about we add cert-manager as part of the published cluster-api components?

[detiber]

@vincepri As long as it wouldn't cause issues with deploying webhooks as part of the cluster-api components I'm good with that solution as well.

[vincepri]

Definitely needs to be tested out, open to any alternative

[vincepri]

/priority important-soon

[tahsinrahman]

@vincepri I think publishing cert-manager as part of clusterapi components(in a single yaml) will not work. As in clusterctl, as all clusterapi components + provider components are passed to clusterctl by single --provider-components flag, but we need to deploy cert-manager separately at first, then deploy other components.
Also, in future if we want to add webhooks for clusterapi types, we need to separate cert-manager from clusterapi components.

We can do the following

$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.10.1/cert-manager.yaml

before deploying clusterapi + provider components

Or, we can copy cert-manager manifests to config/certmanager/cert-manager.yaml in cluster-api, and add a flag --cert-manager-manifests to clusterctl. Providers can use that file in the flag

[vincepri]

Would you mind expanding a little on the reason on why it wouldn't work?

[ncdc]

If we all end up agreeing to proceed with the clusterctl v2 proposal, maybe we defer implementing this until that is closer to being done? And we'd probably want to make deploying cert-manager something the user could opt out of, wouldn't we?

[ncdc]

Moving to the next release
/milestone v0.3.0

[tahsinrahman]

Would you mind expanding a little on the reason on why it wouldn't work?

If we add cert-manager manifests in the same yaml with capi + provider components, it won't work. Until cert-manager pods are not ready, we'll get error

validatingwebhookconfiguration.admissionregistration.k8s.io/capa-validating-webhook-configuration configured
Error from server (InternalError): error when creating "examples/_out/provider-components.yaml": Internal error occurred: failed calling webhook "webhook.certmanager.k8s.io": the server is currently unable to handle the request
Error from server (InternalError): error when creating "examples/_out/provider-components.yaml": Internal error occurred: failed calling webhook "webhook.certmanager.k8s.io": the server is currently unable to handle the request

[detiber]

Is this something that eventually resolves itself? Or does it stay in a perpetual error state?

[detiber]

I'm wondering if we can just dump the yaml and wait out the errors

[tahsinrahman]

I'm wondering if we can just dump the yaml and wait out the errors

yes, eventually cert-manager pods will be ready and it will work

[tahsinrahman]

So, we can publish cert-manager with capi add do a wait.PollImmediate on clusterctl.

[vincepri]

That sounds good to me :)

[munnerz]

If there's anything we in the cert-manager project can do to help here, please let me know and feel free to reach out 😄