Cert-manager: "propagation check failed" from internal cluster

[holmesb]

Describe the bug:
Not sure if this is a bug or expected. Using DNS01 challenge provider from a cluster that only has Internet access via a web proxy (http & https only), we are getting error when enrolling a new cert:
E1108 08:27:38.647486 1 sync.go:184] cert-manager/controller/challenges "msg"="propagation check failed" "error"="dial tcp 64.4.48.5:53: i/o timeout" "dnsName"="host.ourdomain.com" "resource_kind"="Challenge" "resource_name"="host.ourdomain.com-41473054464-4010201230-1730459797" "resource_namespace"="our-namespace" "type"="dns-01"

The IP address of the DNS server being queried (64.4.48.5:53 above) changes on each attempt (every 40 seconds). Our pods cannot query any external DNS servers. Is this a requirement for using Cert Manager?

We have attempted to force this propagation check to use our internal DNS server using the following helm values:

podDnsPolicy: "None"
podDnsConfig:
  nameservers:
    - "10.0.0.1"

where 10.0.0.1 is an internal DNS server our pods can query. While this causes the message that preceeds the error to change from:

I1108 08:03:33.891579 1 dns.go:121] cert-manager/controller/challenges/Check "level"=0 "msg"="checking DNS propagation" "dnsName"="host.ourdomain" "domain"="host.ourdomain.com" "resource_kind"="Challenge" "resource_name"="host.ourdomain.com-41473054464-4010201230-1730459797" "resource_namespace"="our-namepace" "type"="dns-01" "nameservers"=["169.254.25.10:53"]

to

I1108 08:03:33.891579 1 dns.go:121] cert-manager/controller/challenges/Check "level"=0 "msg"="checking DNS propagation" "dnsName"="host.ourdomain.com" "domain"="host.ourdomain.com" "resource_kind"="Challenge" "resource_name"="host.ourdomain.com-41473054464-4010201230-1730459797" "resource_namespace"="our-namespace" "type"="dns-01" "nameservers"=["10.0.0.1:53"]

the same error occurs. Can we use Cert Manager without opening up port 53 access to the world?

Expected behaviour:
Cert is enrolled

Environment details::

• Kubernetes version: 1.15.3
• Cloud-provider/provisioner: bare metal
• cert-manager version: v0.11.0
• Install method (e.g. helm or static manifests): helm

[holmesb]

I was missing switches:
--dns01-recursive-nameservers-only
--dns01-recursive-nameservers="10.0.0.1:53"