Lego: Cloudflare can't find zone for sub-domain

[lexfrei]

I'm trying to issue a certificate for my internal service longhorn.k8s.home.lex.la with traefik and facing the error:

time="2020-11-23T21:05:13Z" level=error msg="Unable to obtain ACME certificate for domains \"longhorn.k8s.home.lex.la\": unable to generate a certificate for the domains [longhorn.k8s.home.lex.la]: error: one or more domains had a problem:\n[longhorn.k8s.home.lex.la] [longhorn.k8s.home.lex.la] acme: error presenting token: cloudflare: failed to find zone k8s.home.lex.la.: Zone could not be found\n" providerName=stagessl.acme routerName=longhorn-ingress-longhorn-system-longhorn-k8s-home-lex-la@kubernetes rule="Host(`longhorn.k8s.home.lex.la`)"

The token rights looks fine:

Looks similar too:
https://github.com/go-acme/lego/issues/1021

Is it my failure or something went wrong on CF's side?

[ldez]

Hello,

the error Zone could not be found comes from Cloudflare, the potential causes are:

• there is no zone for the given zone name
• the zone is not attached to the right AccountID

It doesn't seem related to lego.

https://go-acme.github.io/lego/dns/cloudflare/#api-tokens

[lexfrei]

@ldez, I can issue a certificate for longhorn.lex.la, but can't for longhorn.k8s.home.lex.la with same config. Isn't it proof of a valid token?
I'm trying to find any additional settings/info in CF about this, but I'm a newbie about CF.

[waynema02]

I have a very similar issue, and this only popped up in the last couple of days.

time="2020-11-28T12:21:04-08:00" level=debug msg="legolog: [INFO] [xxx.xxxx.net] acme: Could not find solver for: tls-alpn-01"
time="2020-11-28T12:21:04-08:00" level=debug msg="legolog: [INFO] [xxx.xxxx.net] acme: Could not find solver for: http-01"
time="2020-11-28T12:21:04-08:00" level=debug msg="legolog: [INFO] [xxx.xxxx.net] acme: use dns-01 solver"
time="2020-11-28T12:21:04-08:00" level=debug msg="legolog: [INFO] [xxx.xxxx.net] acme: Preparing to solve DNS-01"
time="2020-11-28T12:21:08-08:00" level=debug msg="legolog: [INFO] [xxx.xxxx.net] acme: Cleaning DNS-01 challenge"
time="2020-11-28T12:21:12-08:00" level=debug msg="legolog: [WARN] [xxx.xxxx.net] acme: cleaning up failed: cloudflare: failed to find zone net.: Zone could not be found "

I'm using traefik as reverse proxy, and all the certs are generated correctly until the last week.

To further narrow down the problem, using the same cloudflare API configuration, I tried both traefik, caddy, and certbot. Neither traefik nor caddy can get a new certificate using dns challenge. However, with the same cloudflare API configuration, certbot actually can get a new certificate.

Thanks

[ldez]

cloudflare: failed to find zone net.: Zone could not be found

@waynema02 it's not the same error. You have an error during the cleaning but it's just a warning.

But from your error message, which not the same as for the issue reporter, I can say that you have an issue with the DNS resolvers.

So you have to override your local resolvers:

• with lego --dns.resolvers
• with Traefik acme.dnsChallenge.resolvers

[waynema02]

@ldez really appreciate the prompt response!

I'll try the check the correct resolvers option, and report back

[waynema02]

@ldez Your suggestion works perfectly!

I just add the resolvers option to my traefik.yaml:

certificatesResolvers:
  myresolver:
    acme:
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - 1.1.1.1:53
          - 8.8.8.8:53
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      storage: "/letsencrypt/acme.json"

And the cert is correctly requested:

time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Obtaining bundled SAN certificate"
time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/163860402"
time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Could not find solver for: tls-alpn-01"
time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Could not find solver for: http-01"
time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: use dns-01 solver"
time="2020-11-28T13:25:28-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Preparing to solve DNS-01"
time="2020-11-28T13:25:32-08:00" level=debug msg="legolog: [INFO] cloudflare: new record for xxxx.xxxx.net, ID xxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
time="2020-11-28T13:25:32-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Trying to solve DNS-01"
time="2020-11-28T13:25:32-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Checking DNS record propagation using [1.1.1.1:53 8.8.8.8:53]"
time="2020-11-28T13:25:34-08:00" level=debug msg="legolog: [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]"
time="2020-11-28T13:25:46-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] The server validated our request"
time="2020-11-28T13:25:46-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Cleaning DNS-01 challenge"
time="2020-11-28T13:25:46-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] acme: Validations succeeded; requesting certificates"
time="2020-11-28T13:25:49-08:00" level=debug msg="legolog: [INFO] [xxxx.xxxx.net] Server responded with a certificate."

In retrospect, I did setup a pihole a few days ago, and pointed all my servers to it. Probably since then I cannot get any new certificate. Maybe somehow the pihole interfered with the FQDN resolve? That's a bit weird.. Anyway, getting all the DNS pieces right is always a struggle for me :(

My issue is fully resolved. Thanks!

[lexfrei]

@ldez look at this:
(gist)

$ go run . <TOKEN>
2020/11/29 02:07:43 trying to login...
2020/11/29 02:07:43 logged in!

2020/11/29 02:07:43 trying to get zone ID like lego tries to do so...
2020/11/29 02:07:43 k8s.home.lex.la is in use
2020/11/29 02:07:46 Ofc, it's not working: Zone could not be found

2020/11/29 02:07:46 trying to get zone ID by short domain...
2020/11/29 02:07:47 lex.la is in use
2020/11/29 02:07:47 zone id is: 8e3fd4280829357a4081156b0028442f

2020/11/29 02:07:47 trying to create TXT in this zone...
2020/11/29 02:07:48 done!

2020/11/29 02:07:48 trying retrieve TXT by ID...
2020/11/29 02:07:50 longhorn.k8s.home.lex.la    TXT     hi, lego!
:)

Feel free to test it with dig -t txt longhorn.k8s.home.lex.la

Looks like lego should use just a second-level domain to retrieve zone id and trim the domain, not like this

I'm not familiar with CF's API so, I can't create a PR with fix by myself, sorry.

[ldez]

I think there is no problem with lego.

Have you tried the solution I suggested to waynema02?

https://github.com/go-acme/lego/issues/1303#issuecomment-735289513

I think you're dealing with a DNS cache problem or a misconfiguration of your zone.

[lexfrei]

@ldez oh. My bad. Just tried the lego itself and it works!
But, there is a way how I can reproduce the traefik's Zone could not be found error for domains with more than 3 levels.
Let me investigate it a little more and I'll close this issue.
Sorry for blaming the lego.