Cert-manager: challenge stuck on pending dial tcp: lookup <traefik host> on <kube-dns svc ip>: no such host

[qwertzxcvb]

issuer.yaml

apiVersion: certmanager.k8s.io/v1alpha1
   kind: ClusterIssuer
   metadata:
     name: letsencrypt-prod
   spec:
     acme:
       # The ACME server URL
       server: https://acme-v02.api.letsencrypt.org/directory
       # Email address used for ACME registration
       email: [email protected]
       # Name of a secret used to store the ACME account private key
       privateKeySecretRef:
         name: letsencrypt-prod
       # Enable the HTTP-01 challenge provider
       #http01: {}
       solvers:
       - selector: {}
         http01:
           ingress:
             class: traefik

ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: "traefik"
    certmanager.k8s.io/cluster-issuer: "letsencrypt-prod"
    certmanager.k8s.io/acme-challenge-type: http01
spec:
  tls:
  - hosts:
    - omt.traefikui.com
    secretName: traefik-ui
  rules:
  - host: omt.traefikui.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: web

challenge

Name:         traefik-ui-2866269153-0
Namespace:    kube-system
Labels:       acme.cert-manager.io/order-name=traefik-ui-2866269153
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Challenge
Metadata:
  Creation Timestamp:  2019-06-11T06:12:34Z
  Finalizers:
    finalizer.acme.cert-manager.io
  Generation:  4
  Owner References:
    API Version:           certmanager.k8s.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Order
    Name:                  traefik-ui-2866269153
    UID:                   e58c9408-8c0f-11e9-b4c5-049fca8e1048
  Resource Version:        1239884
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/kube-system/challenges/traefik-ui-2866269153-0
  UID:                     e786e737-8c0f-11e9-b4c5-049fca8e1048
Spec:
  Authz URL:  https://acme-v02.api.letsencrypt.org/acme/authz/8Q6gzhwzW00S_hYHJMUHW-DF0OIq4-i3GZcIHeJ89-c
  Dns Name:   omt.traefikui.com
  Issuer Ref:
    Kind:  ClusterIssuer
    Name:  letsencrypt-prod
  Key:     jcMBYJwqIvQlnm8y2a6dBneFRFonJetkDkH93oWwF4U.sq-eztnhYteof8-DDJrB0q0Me-qB7TYlOpuloxPi2Vo
  Solver:
    Http 01:
      Ingress:
        Class:  traefik
    Selector:
  Token:     jcMBYJwqIvQlnm8y2a6dBneFRFonJetkDkH93oWwF4U
  Type:      http-01
  URL:       https://acme-v02.api.letsencrypt.org/acme/challenge/8Q6gzhwzW00S_hYHJMUHW-DF0OIq4-i3GZcIHeJ89-c/16921018933
  Wildcard:  false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://omt.traefikui.com/.well-known/acme-challenge/jcMBYJwqIvQlnm8y2a6dBneFRFonJetkDkH93oWwF4U': Get http://omt.traefikui.com/.well-known/acme-challenge/jcMBYJwqIvQlnm8y2a6dBneFRFonJetkDkH93oWwF4U: dial tcp: lookup omt.traefikui.com on 10.96.0.10:53: no such host
  State:       pending
Events:        <none>

it seems like to lookup the traefik-ingress-host in kube-dns service！！Of course it does not exist

kubernetes 1.14.2
cert-manager 0.8.0

[munnerz]

it seems like to lookup the traefik-ingress-host in kube-dns service！！Of course it does not exist

So this is as expected, and it should exist. cert-manager does not create DNS A records for you, it is your job to create the A record for omt.traefikui.com pointing at the IP address of your Ingress controller.

[munnerz]

I'm going to close this as I think your configuration is incorrect. The #cert-manager channel on slack.k8s.io is helpful for attempting to debug your config before opening an issue about it, so that we can keep this issue board for bug reports and feature requests 😄

Feel free to reopen if you think there is still a bug in cert-manager that can be fixed here!

[bappr]

@munnerz actually I am getting the same issue. And I am quite sure of the error since it's a "no such host" error and I can reach it easily.

E1005 11:35:20.883814       1 sync.go:183] cert-manager/controller/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://kintocloud.com/.well-known/acme-challenge/AdTZppUutYuiH_WA9kHV56l4mufmbVmuZE41LAUTRas': Get http://kintocloud.com/.well-known/acme-challenge/AdTZppUutYuiH_WA9kHV56l4mufmbVmuZE41LAUTRas: dial tcp: lookup kintocloud.com on 10.40.0.10:53: no such host" "dnsName"="kintocloud.com" "resource_kind"="Challenge" "resource_name"="kintocloud-cert-1072522747-0" "resource_namespace"="istio-system" "type"="http-01"

http://kintocloud.com/.well-known/acme-challenge/AdTZppUutYuiH_WA9kHV56l4mufmbVmuZE41LAUTRas

EDIT:
I removed the URL so not accessible anymore.
Still getting hit by this issue when the DNS is not configured before the cert is created.
Restarting cert-manager does not even fix the problem.

I resolves after sometime (not so sure if it's always the case) so wondering if there is any TTL configured somewhere? Definitely not on my DNS.

[kushwahashiv]

Hi,

Im gettign following error for my GKE cluster. My domain is registered in godaddy.com
My question what is causing below error and is this something I need to do on gopdaddy.com so that these well-known thing can be verified

cert-manager/controller/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://www.abc.com/.well-known/acme-challenge/AZl_evY1PscNKi95EdfFNuYG_Gl75-Hi8we7Efbyy7I': Get http://www.abc/.well-known/acme-challenge/AZl_evY1PscNKi95E3EFNxcdfl75-Hi8we7Efbyy7I: dial tcp: lookup www.abc.in on 10.12.244.10:53: no such host" "dnsName"="www.abc.com" "resource_kind"="Challenge" "resource_name"="abc-3611830638-3386842356-1600275291" "resource_namespace"="default" "type"="http-01"

[alemonmk]

I believe anyone who use same DNS zone for cluster internal suffix and exposed service (i.e. you use k8s.example.com and exposing a service via /(app.)?k8s.example.com/) will be hit by this, and I would not say it's configuration issue. Let's see if #1292 ever got fixed ...
Hmm... if this is the use case and you are using CoreDNS, you can just edit its Corefile, making kubernetes.fallthrough to apply on all zones.

[EternalDeiwos]

@alemonmk I don't know why you crossed that text out -- that was exactly my problem.

[alemonmk]

@EternalDeiwos Actually I found the proper way: uncomment these and point to any server that can resolve your names.