Cert-manager: "Internal error occurred: failed calling webhook" GKE Private Cluster

[Berndinox]

Describe the bug:

kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.2/cert-manager.yaml

kubectl apply -f clusterissuer.yaml

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: _MYMAIL_
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - dns01:
        cloudflare:
          email: _MYMAIL_
          apiKeySecretRef:
            name: cloudflare-apikey-secret
            key: apikey

kubectl apply -f ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/acme-challenge-type: dns01
    cert-manager.io/acme-dns01-provider: cloudflare
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: 'true'
  name: wp-bernd-ingress
  namespace: wp-bernd
spec:
  rules:
  - host: _MYHOST_
    http:
      paths:
      - backend:
          serviceName: wp-bernd-klaus-onl-svc
          servicePort: 80
  tls:
  - hosts:
    - _MYHOST_
    secretName: wp-bernd-tls

Error from server (InternalError): error when creating "ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io
": Post https://ingress-nginx-controller-admission.ingress-nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s: net/http: request canceled while waiting for co
nnection (Client.Timeout exceeded while awaiting headers)



md5-e51576f4f7815729920cda0189ee3e2f



v1alpha2.acme.cert-manager.io          Local                        True        17m
v1alpha2.cert-manager.io               Local                        True        17m
v1alpha3.acme.cert-manager.io          Local                        True        17m
v1alpha3.cert-manager.io               Local                        True        17m



md5-789a35e3277d7d8cf91e5e07e1181fde



NAME                                      READY   STATUS    RESTARTS   AGE
cert-manager-5d9cd85cbb-4jxpp             1/1     Running   0          18m
cert-manager-cainjector-95c885477-878ls   1/1     Running   0          18m
cert-manager-webhook-6ff9487489-d6vdl     1/1     Running   0          18m

Environment details::
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.4-gke.10"
Cert-Manager 14.2
Install without Helm

/kind bug

[meyskens]

failed calling webhook "validate.nginx.ingress.kubernetes.io indicates an issue in the Nginx Ingress component, not with cert-manager.

/triage support

[Berndinox]

hy @meyskens - thanks for the fast reposne, ... ofc, have not tought about this. will look into it..

For reference, start here: https://github.com/kubernetes/ingress-nginx/issues/5401

Adding Port 8443 fixed it

/close