Cert-manager: Internal error occurred: failed calling webhook. Wrong cert-manager hook ID.

Created on 3 Dec 2019  Â·  14Comments  Â·  Source: jetstack/cert-manager

Try to apply issuer to kubernetes and get this error:
Error from server (InternalError): error when creating ".\\issuer.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-1575287841-webhook.cert-manager.svc:443/mutate?timeout=30s: service "cert-manager-1575287841-webhook" not found

Seems it try to call this "cert-manager-1575287841-webhook", but in my kubernetes i have "cert-manager-1575353985-webhook".

Most helpful comment

have the same issue

Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: context deadline exceeded

All 14 comments

Had the exact same issue as you. Were running an older version (v0.5.0) - upgrades were failing. Decided to update to v. 0.12.0 (helm charts) and received your listed error.

After backup and helm chart cert-manager deleted and purged I kept getting the error with reference to an old webhook.
I manually went through the below CRDs and removed old references before reinstalling.

v1/ClusterRole
v1/Deployment
v1/Pod(related)
v1/Service
v1/ServiceAccount
v1beta1/ClusterRole
v1beta1/ClusterRoleBinding
v1beta1/MutatingWebhookConfiguration
v1beta1/Role
v1beta1/RoleBinding
v1beta1/ValidatingWebhookConfiguration

I'm running on AKS so I've actually also updated all nodes to newst version trying to get the cluster to forget the old references - not sure this actually helped though.

Hope this helps you.

have the same issue. also on AKS from azure. only an update to the newest version fixed my problem

have the same issue

Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: context deadline exceeded

Also happens on the latest version v0.13.0 for me:

$ kubectl apply -f test-resources.yaml                                                                                           
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://eve-dev-cert-manager-webhook.eve.svc:443/mutate?timeout=30s: service "eve-dev-cert-manager-webhook" not found
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://eve-dev-cert-manager-webhook.eve.svc:443/mutate?timeout=30s: service "eve-dev-cert-manager-webhook" not found

@robinmanuelthiel service "eve-dev-cert-manager-webhook" not found Did you update the name and namespace for this in all the manifests?

I had a similar error to this, but had to update the following annotation:

annotations:
    cert-manager.io/inject-ca-from-secret: "lb/cert-manager-webhook-tls"

have the same issue

Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager2-webhook.cert-manager2.svc:443/mutate?timeout=30s: service "cert-manager2-webhook" not found

Did you rename the Service? Looks like it should be name: cert-manager2-webhook

Did you rename the Service? Looks like it should be name: cert-manager2-webhook

I followed the documentation(https://cert-manager.io/docs/installation/kubernetes/) and so I didn't rename it. How to do this?, I am a beginner.

I'd look for the YAML object that has kind: Service in it.

It takes a couple times for it to take

TASK [Install ACME Issuer] ********************************************************************************************************************
FAILED - RETRYING: Install ACME Issuer (10 retries left).
FAILED - RETRYING: Install ACME Issuer (9 retries left).
changed: [localhost] => {"attempts": 3, "changed": true, "cmd": "kubectl apply -f acme-issuer.yaml", "delta": "0:00:00.765282", "end": "2020-03-03 07:35:59.633997", "rc": 0, "start": "2020-03-03 07:35:58.868715", "stderr": "", "stderr_lines": [], "stdout": "clusterissuer.cert-manager.io/letsencrypt-staging created", "stdout_lines": ["clusterissuer.cert-manager.io/letsencrypt-staging created"]}

https://github.com/TheShellLand/antsable/blob/master/playbooks/rancher2-singlenode.yml#L143-L148

Hi guys i've got the same issue after following this tutorial

namespace/cert-manager-test unchanged
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: Service Unavailable
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: Service Unavailable

Cluster status below

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-579d48dff8-86fl9              1/1     Running   0          168m
cert-manager-cainjector-789955d9b7-x7b29   1/1     Running   0          168m
cert-manager-webhook-64869c4997-l752v      1/1     Running   0          168m

I tried to follow instructions here but there is some information missing for my understanding. (I'm a beginner)

Ξ /tmp/HELM → k exec -n kube-system kube-apiserver-nodeserver -- kube-apiserver --enable-admission-plugins=MutatingAdmissionWebhook,ValidatingAdmissionWebhook


W0326 19:40:53.844980     133 services.go:37] No CIDR for service cluster IPs specified. Default value which was 10.0.0.0/24 is deprecated and will be removed in future releases. Please specify it using --service-cluster-ip-range on kube-apiserver.
I0326 19:40:53.845058     133 server.go:596] external host was not specified, using 172.24.112.36
W0326 19:40:53.845066     133 authentication.go:439] AnonymousAuth is not allowed with the AlwaysAllow authorizer. Resetting AnonymousAuth to false. You should use a different authorizer

Find below my configurations:

  • Kubernetes v1.17.3
  • Cert-manager v0.14.1

Please, if you can give me some help on that issue I'll appreciate.

Regards.

Issues like this are ultimately caused by a mismatch between the webhook's service name configured on the CRD resources (which are not templated as part of the Helm chart, and thus are not updated automatically if the release name is set to something other than cert-manager or if the namespace is also not cert-manager).

In the v0.15 releases (for which you can fetch the alpha.1 release now), we have introduced the installCRDs option to the Helm chart. When used either with helm install or helm template, this should resolve the issues here automatically and correctly set the release name and namespace on all resources accordingly.

Because we've now got this new option, I think we can close this issue. For those running versions earlier than v0.15 in their clusters, you'll need to ensure to correctly replace all references to the cert-manager namespace and cert-manager-webhook Service resource in the CRDs when applying them. This includes in the spec.conversion and the metadata.annotations sections.

i follow this guide to install it, https://cert-manager.io/docs/installation/kubernetes/, but still had the same issue

[root@master1 cert-manger]# kubectl get pods --namespace cert-manager -o wide
NAME                                      READY   STATUS    RESTARTS   AGE     IP            NODE      NOMINATED NODE   READINESS GATES
cert-manager-7747db9d88-kzq4x             1/1     Running   0          2m58s   10.244.6.21   master2   <none>           <none>
cert-manager-cainjector-87c85c6ff-bhllv   1/1     Running   0          2m58s   10.244.1.22   node1     <none>           <none>
cert-manager-webhook-64dc9fff44-g6q8f     1/1     Running   0          17s     10.244.6.22   master2   <none>           <none>
[root@master1 cert-manger]# kubectl apply -f test-resources.yaml
namespace/cert-manager-test unchanged



Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: context deadline exceeded
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: context deadline exceeded

I've installed with 0.15.2 with the installCRDs option and this is still happening for me as well. Not sure where to go from here

Was this page helpful?
0 / 5 - 0 ratings

Related issues

jakubknejzlik picture jakubknejzlik  Â·  3Comments

cpick picture cpick  Â·  3Comments

Azylog picture Azylog  Â·  3Comments

munjal-patel picture munjal-patel  Â·  3Comments

matthew-muscat picture matthew-muscat  Â·  4Comments