Caddy: Caddy fails to retrieve aws challenge txt record value

Created on 8 Jun 2020 · 4Comments · Source: caddyserver/caddy

Hello Everybody!
I'd like to ask for help regarding setting up caddy as a docker container to supply an endpoint with a valid letsencrypt certificate as a reverse proxy.

First I build a caddy 2.0.0 container with the aws route53 provider.
This is my Caddyfile:

{
  debug
  email [email protected]
}

my.domain.com {
  reverse_proxy https://192.168.1.1:8443 {
       transport http {
           tls
           tls_insecure_skip_verify
       }
  }
  tls {
    dns route53 {
      max_retries 2
    }
  }
}

This is my docker-compose file for caddy:

  caddy:
    container_name: caddy
    image: caddy-route53:2.0.0
    restart: unless-stopped
    hostname: caddy
    ports:
      - "192.168.1.21:2019:2019"
      - "192.168.1.21:80:80"
      - "192.168.1.21:443:443"
    environment:
      AWS_ACCESS_KEY_ID: 'someId'
      AWS_SECRET_ACCESS_KEY: 'someKey'
      AWS_HOSTED_ZONE_ID: 'someZoneId'
    volumes:
       - './caddy/Caddyfile:/etc/caddy/Caddyfile'
       - './caddy/resolv.conf:/etc/resolv.conf'
    #   - 'caddy/certs/:/root/.caddy'
    dns:
      - 1.1.1.1
      - 1.0.0.1
      - 8.8.8.8
      - 8.4.4.8
    cap_add:
      - NET_ADMIN

Caddy boots up and successfully deployes the record inside my dns domain at aws.
Then it tries to retrieve the record and fails:

...
2020/06/08 18:26:44 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:46 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:48 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:50 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:53 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:55 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:57 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:26:59 [INFO] [my.domain.com] acme: Waiting for DNS record propagation.
2020/06/08 18:27:01 [INFO] [my.domain.com] acme: Cleaning DNS-01 challenge
2020/06/08 18:28:04 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5104633946
2020/06/08 18:28:05 [ERROR] error: one or more domains had a problem:
[unifi.karstensiemer.de] time limit exceeded: last error: NS ns-558.awsdns-05.net. did not return the expected TXT record [fqdn: _acme-challenge.my.domain.com., value: value]:
 (challenge=dns-01 remaining=[])
2020/06/08 18:28:07 [ERROR] attempt 1: [my.domain.com] Obtain: [my.domain.com] error: one or more domains had a problem:
[my.domain.com] time limit exceeded: last error: NS ns-558.awsdns-05.net. did not return the expected TXT record [fqdn: _acme-challenge.my.domain.com., value: value]:
 - retrying in 1m0s (3m14.100346839s/720h0m0s elapsed)...
...

The aws record is absolutely correct and in the way I'd expect it.

What I have tried to fix this:

At first I didn't supply any extra dns settings via docker. It failed.
I tried supplying dns servers via docker-compose (those in the docker-compose file). Caddy still used 127.0.0.11 because thats the way docker works. But the docker internal resolver should have used those. It failed.
I execed into the container and installed bind-tools and did a dig on the record while caddy failed to retrieve it, and I could successfully resolve the record via the dns servers caddy asked.
I swapped the resolv.conf from caddys container with one where the dns servers where in directly.
Inside caddys log it showed that it tries to connect to them directly, bypassing the docker internal one, still caddy failed with retrieving the record.

One maybe important note to make here, my.domain.com is of course not my real domain.
The one I am trying to get a cert for is actually a thrid level domain like this.my.domain.com
my.domain.com is hosted at a other provider than aws but this.my.domain.com is delegated to aws where I already created an A record for it. But this shoudn't hinder caddy from getting a cert should it?

I'd be super happy if any of you could asist me.
Thanks for taking the Time!

question

Source

KarstenSiemer

All 4 comments

Hi! Thanks for trying Caddy!

Please ask your usage questions on the Caddy community forums. We prefer to keep the GitHub issue board for bugs and feature requests. Don't forget to fill out the thread template so we can help you!

At a quick glance, it looks like you're missing the caddy_data:/data volume, that's where Caddy stores its certificate data now. It's _very_ important to persist that.

francislavoie on 8 Jun 2020

If this turns out to be a bug, the right place to make an issue would be here: https://github.com/caddy-dns/route53. But until you know it's a bug, it's probably best to ask the question on the forums. Thanks!

francislavoie on 8 Jun 2020

Actually that package is only a few line wrapper to make https://github.com/libdns/route53 into a Caddy module. If there's a bug, it's probably in the libdns package.