Charts: [stable/cert-manager] v0.6.0 fails to install w/ missing webhook tls secret

[haf]

Is this a request for help?: No and yes. I would prefer not to have to fix this bug myself.
Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug
Version of Helm and Kubernetes:

Client: &version.Version{SemVer:"v2.12.2", GitCommit:"7d2b0c73d734f6586ed222a567c5d103fed435be", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.2", GitCommit:"7d2b0c73d734f6586ed222a567c5d103fed435be", GitTreeState:"clean"}

Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.6-gke.3", GitCommit:"04ad69a117f331df6272a343b5d8f9e2aee5ab0c", GitTreeState:"clean", BuildDate:"2019-01-10T00:39:15Z", GoVersion:"go1.10.3b4", Compiler:"gc", Platform:"linux/amd64"}

Which chart: cert-manager /cc @munnerz
What happened:

$ k describe pods stage-cert-manager-webhook-666b98fb85-lqgz2
...

  Type     Reason       Age                  From                                                       Message
  ----     ------       ----                 ----                                                       -------
  Normal   Scheduled    11m                  default-scheduler                                          Successfully assigned kube-system/stage-cert-manager-webhook-666b98fb85-lqgz2 to gke-stage-cluster-default-pool-809f
  Warning  FailedMount  85s (x13 over 11m)   kubelet, gke-stage-cluster-default-pool-809f  MountVolume.SetUp failed for volume "certs" : secrets "stage-cert-manager-webhook-webhook-tls" not found
  Warning  FailedMount  36s (x5 over 9m39s)  kubelet, gke-stage-cluster-default-pool-809f  Unable to mount volumes for pod "stage-cert-manager-webhook-666b98fb85-lqgz2_kube-system(dd2b3df8-1f58-11e9-b713-42010a9c0180)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"stage-cert-manager-webhook-666b98fb85-lqgz2". list of unmounted volumes=[certs]. list of unattached volumes=[certs stage-cert-manager-webhook-token-kxfpc]

Commands to repro:

  # curl -o cert-manager-crds.yaml -L https://raw.githubusercontent.com/jetstack/cert-manager/release-0.6/deploy/manifests/00-crds.yaml
  kubectl apply --namespace kube-system \
    -f cert-manager-crds.yaml

  helm install --namespace kube-system \
    --name "${ENVIRONMENT}-cert-manager" \
    --version "v0.6.0" \
    stable/cert-manager

What you expected to happen: The core components of cert-manager should be successfully installed

How to reproduce it (as minimally and precisely as possible): See above

Anything else we need to know:

[munnerz]

If you're installing the chart into an existing namespace, you'll need to ensure that namespace has an additional label in order to ensure the chart can be installed properly (as per https://cert-manager.readthedocs.io/en/latest/admin/resource-validation-webhook.html#tls-configuration).

We should probably update the chart README to explicitly call this out though!

[haf]

The docs state that

This is handled automatically when performing a helm install for the first time by use of an additional selector in the ValidatingWebhookConfiguration

But when actually verifying this (and not providing an explicit namespace to helm install, it is not true:

$ kubectl describe ns cert-manager
Name:         cert-manager
Labels:       name=cert-manager
Annotations:  <none>
Status:       Active

Resource Quotas
 Name:                       gke-resource-quotas
 Resource                    Used  Hard
 --------                    ---   ---
 count/ingresses.extensions  0     1G
 count/jobs.batch            1     1G
 pods                        2     1G
 services                    1     1G

No resource limits.

Are you saying that --namespace something-else causes the special-case webhook to succeed despite there being no namespace label? Or are you saying that cert-manager is a special-case namespace? Or are you saying that the fact that I manually labelled cert-manager directly after installing the chart caused this chart to succeed?

[gregkeys]

I got the same error, new installation of minikube, followed the instructions in the readme and got this error MountVolume.SetUp failed for volume "certs" : secrets "cert-manager-webhook-webhook-tls" not found

[jurgenweber]

@gregkeys I resolved that error with this; https://github.com/jetstack/cert-manager/pull/1259/files and killing/deleting the pods. I have no errors now.

[gregkeys]

I get the error deploying into minikube's default namespace, will try adding the label to see if that helps

[cay914]

@jurgenweber Your answer is right.
I delete the cert-manager and reinstall it.
It works.

IMPORTANT: if you are deploying into a namespace that *already exists,
you MUST ensure the namespace has an additional label on it in order for
the deployment to succeed*

$ kubectl apply \
-f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.6/deploy/manifests/00-crds.yaml
$ kubectl label namespace certmanager.k8s.io/disable-validation="true"
$ helm install --name my-release stable/cert-manager

[haf]

@cay914 Wouldn't that cause re-issuing of all certificates, as even the certificate user-define type would be deleted?

[todinov]

I installed with the label in place, but still get the same error in GKE. It works locally on minikube. Any ideas?

EDIT: I removed the namespace and created it again with the manifest file (https://raw.githubusercontent.com/jetstack/cert-manager/release-0.6/deploy/manifests/01-namespace.yaml) and it worked after that. Not sure why the one-liner failed for me.