Helm-charts: [kube-prometheus-stack] prometheusOperator.admissionWebhooks.enabled=false fails

[ansd]

Describe the bug

helm install prom prometheus-community/kube-prometheus-stack \
  --set prometheusOperator.admissionWebhooks.enabled=false

results in

pod/prom-kube-prometheus-stack-operator-869f7cfd69-2nfzr   0/1     ContainerCreating   0          2m14

and

Warning  FailedMount  26s (x9 over 2m34s)  kubelet            MountVolume.SetUp failed for volume "tls-secret" : secret "prom-kube-prometheus-stack-admission" not found

Version of Helm and Kubernetes:

Helm Version:

$ helm version
version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"dirty", GoVersion:"go1.15.4"

Kubernetes Version:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.10-gke.2101", GitCommit:"abb34dc6bb651ef0eb9d25d868a5a8b9001201ff", GitTreeState:"clean", BuildDate:"2020-11-06T18:24:35Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}

Which chart:
kube-prometheus-stack

Which version of the chart:
12.2.3

What happened:

> kubectl describe pod/prom-kube-prometheus-stack-operator-869f7cfd69-2nfzr
...
Volumes:
  tls-secret:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  prom-kube-prometheus-stack-admission
    Optional:    false
...
Events:
  Type     Reason       Age                  From               Message
  ----     ------       ----                 ----               -------
  Normal   Scheduled    2m34s                default-scheduler  Successfully assigned default/prom-kube-prometheus-stack-operator-869f7cfd69-2nfzr to gke-lre-default-pool-8af3bc28-6mj6
  Warning  FailedMount  31s                  kubelet            Unable to attach or mount volumes: unmounted volumes=[tls-secret], unattached volumes=[prom-kube-prometheus-stack-operator-token-c52dz tls-secret]: timed out waiting for the condition
  Warning  FailedMount  26s (x9 over 2m34s)  kubelet            MountVolume.SetUp failed for volume "tls-secret" : secret "prom-kube-prometheus-stack-admission" not found

What you expected to happen:
Prometheus operator pod doesn't hang in status ContainerCreating.

How to reproduce it (as minimally and precisely as possible):

helm install prom prometheus-community/kube-prometheus-stack \
  --set prometheusOperator.admissionWebhooks.enabled=false

The helm command that you execute and failing/misfunctioning:
same as previous command

Helm values set after installation/upgrade:

> helm get values prom
USER-SUPPLIED VALUES:
prometheusOperator:
  admissionWebhooks:
    enabled: false

Additional context:
Might be related to https://github.com/prometheus-community/helm-charts/blob/021fa0ebae5b523493be99b951b5fd4738e2bb74/charts/kube-prometheus-stack/templates/prometheus-operator/deployment.yaml#L116-L122
However, the secret doesn't get created if prometheusOperator.admissionWebhooks.enabled=false is set.

[sentros]

I'm experiencing this exact same problem.

[jetersen]

according the lines below you need to add prometheusOperator.tls.enabled
https://github.com/prometheus-community/helm-charts/blob/63e0c0ef24c41a0849daf937c9db3995c16b52a5/charts/kube-prometheus-stack/templates/prometheus-operator/deployment.yaml#L118-L124

So this is required:

prometheusOperator:
  tls:
    enabled: false
  admissionWebhooks:
    enabled: false

[alexppg]

Why this change? Why you suddenly need a tls secret?

[MythicManiac]

I'm having a hard time finding documentation on how are you supposed to configure the TLS in this chart, and as I'm not using helm for the installation, the steps described in https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#how-the-chart-configures-the-hooks are not very useful.

As I understand it, I need a self signed certificate and CA. I however do not know where exactly are these supposed to be placed and in what format. The values.yaml file being 2.3k lines long and lack of documentation doesn't exactly help.

EDIT: After some poking around in the chart, I found the prometheusOperator.admissionWebhooks.certManager.enabled flag, which if set to true, will automatically provision a self-signed certificate by using cert-manager, given that cert-manager is installed on the cluster.

EDIT2: I've opened #574 to address the lack of documentation