Caddy: Caddy API adds new host to the route matcher, but does not issue a certificate.
I am trying to use the Caddy API and add new domain string to the host array within a route->match json node using the command below:
curl -X POST \
-H "Content-Type: application/json" \
-d '"www.istand.ie"' \
"http://localhost:2019/config/apps/http/servers/srv0/routes/0/match/0/host/0"
It seems that the new domain is added successfully to the array, but I get error panic: certificate worker: runtime error: slice bounds out of range. and new cert is not issued for that domain.
Not sure if this is because I am using plugin for the cert store or if I am doing something wrong.
Here is the output:
ice1@ice2:~/ice1$ curl -X POST \
> -H "Content-Type: application/json" \
> -d '"www.istand.ie"' \
> "http://localhost:2019/config/apps/http/servers/srv0/routes/0/match/0/host/0"
2020/12/27 16:18:08.072 INFO admin.api received request {"method": "POST", "host": "localhost:2019", "uri": "/config/apps/http/servers/srv0/routes/0/match/0/host/0", "remote_addr": "127.0.0.1:34674", "headers": {"Accept":["*/*"],"Content-Length":["15"],"Content-Type":["application/json"],"User-Agent":["curl/7.68.0"]}}
2020/12/27 16:18:08.073 INFO admin admin endpoint started {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
1.6090858880738919e+09 info tls.cache.maintenance started background certificate maintenance {"cache": "0xc0002c1b20"}
2020/12/27 16:18:08.073 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0xc0002c1b20"}
2020/12/27 16:18:08.073 INFO http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
1.6090858880739849e+09 info http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2020/12/27 16:18:08.074 INFO http enabling automatic HTTP->HTTPS redirects {"server_name": "srv0"}
1.6090858880740697e+09 info http enabling automatic HTTP->HTTPS redirects {"server_name": "srv0"}
2020/12/27 16:18:08.074 INFO http server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server {"server_name": "srv1", "http_port": 80}
1.6090858880741022e+09 info http server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server {"server_name": "srv1", "http_port": 80}
2020/12/27 16:18:08.074 WARN http user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects {"server_name": "srv1", "interface": "tcp/:80"}
1.6090858880742252e+09 warn http user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects {"server_name": "srv1", "interface": "tcp/:80"}
1.6090858880746253e+09 debug http starting server loop {"address": "[::]:443", "http3": false, "tls": true}
1.60908588807466e+09 debug http starting server loop {"address": "[::]:80", "http3": false, "tls": false}
2020/12/27 16:18:08.074 INFO http enabling automatic TLS certificate management {"domains": ["www.istand.ie", "corner.ie", "icewolf.app", "istand.ie", "*.corner.ie", "www.icewolf.app"]}
1.6090858880746655e+09 info http enabling automatic TLS certificate management {"domains": ["www.istand.ie", "corner.ie", "icewolf.app", "istand.ie", "*.corner.ie", "www.icewolf.app"]}
2020/12/27 16:18:08 panic: certificate worker: runtime error: slice bounds out of range [3:2]
goroutine 660 [running]:
github.com/caddyserver/certmagic.(*jobManager).worker.func1()
github.com/caddyserver/[email protected]/async.go:58 +0x9e
panic(0x18c7280, 0xc0007a4360)
runtime/panic.go:969 +0x1b9
github.com/caddyserver/certmagic.(*ACMEManager).mostRecentAccountEmail(0xc000bee000, 0x1a33894, 0x2e, 0x0, 0x0, 0x0)
github.com/caddyserver/[email protected]/account.go:320 +0x6a5
github.com/caddyserver/certmagic.(*ACMEManager).getEmail(0xc000bee000, 0x0, 0x19f1401, 0x13)
github.com/caddyserver/[email protected]/account.go:171 +0x165
github.com/caddyserver/certmagic.(*ACMEManager).PreCheck(0xc000bee000, 0x1cee840, 0xc000be2380, 0xc000a353c0, 0x1, 0x1, 0x0, 0x0, 0x0)
github.com/caddyserver/[email protected]/acmemanager.go:213 +0xdf
github.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).PreCheck(0xc0000c4500, 0x1cee840, 0xc000be2380, 0xc000a353c0, 0x1, 0x1, 0x40e300, 0x10, 0x1725520)
github.com/caddyserver/caddy/[email protected]/modules/caddytls/acmeissuer.go:187 +0x116
github.com/caddyserver/certmagic.(*Config).getPrecheckedIssuer(0xc000bb85a0, 0x1cee840, 0xc000be2380, 0xc000a353c0, 0x1, 0x1, 0x1918300, 0xc000064800, 0x0, 0xd, ...)
github.com/caddyserver/[email protected]/config.go:710 +0x179
github.com/caddyserver/certmagic.(*Config).ObtainCert(0xc000bb85a0, 0x1cee840, 0xc000be2380, 0xc0008c1940, 0xd, 0x0, 0xc00005dcd8, 0x40efcd)
github.com/caddyserver/[email protected]/config.go:378 +0xd6
github.com/caddyserver/certmagic.(*Config).manageOne.func1(0x7207dd, 0xc0007ac1c0)
github.com/caddyserver/[email protected]/config.go:310 +0x95
github.com/caddyserver/certmagic.(*jobManager).worker(0x2731c20)
github.com/caddyserver/[email protected]/async.go:73 +0x103
created by github.com/caddyserver/certmagic.(*jobManager).Submit
github.com/caddyserver/[email protected]/async.go:50 +0x12d
2020/12/27 16:18:08 panic: certificate worker: runtime error: slice bounds out of range [3:2]
goroutine 661 [running]:
github.com/caddyserver/certmagic.(*jobManager).worker.func1()
github.com/caddyserver/[email protected]/async.go:58 +0x9e
panic(0x18c7280, 0xc000969b80)
runtime/panic.go:969 +0x1b9
github.com/caddyserver/certmagic.(*ACMEManager).mostRecentAccountEmail(0xc00024e9c0, 0x1a33894, 0x2e, 0x0, 0x0, 0x0)
github.com/caddyserver/[email protected]/account.go:320 +0x6a5
github.com/caddyserver/certmagic.(*ACMEManager).getEmail(0xc00024e9c0, 0x0, 0x19f1401, 0x13)
github.com/caddyserver/[email protected]/account.go:171 +0x165
github.com/caddyserver/certmagic.(*ACMEManager).PreCheck(0xc00024e9c0, 0x1cee840, 0xc000be2380, 0xc0008e4850, 0x1, 0x1, 0x0, 0x0, 0x0)
github.com/caddyserver/[email protected]/acmemanager.go:213 +0xdf
github.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).PreCheck(0xc0000c4500, 0x1cee840, 0xc000be2380, 0xc0008e4850, 0x1, 0x1, 0x40e300, 0x10, 0x1725520)
github.com/caddyserver/caddy/[email protected]/modules/caddytls/acmeissuer.go:187 +0x116
github.com/caddyserver/certmagic.(*Config).getPrecheckedIssuer(0xc000bb85a0, 0x1cee840, 0xc000be2380, 0xc0008e4850, 0x1, 0x1, 0x1918300, 0x0, 0x19d9d92, 0xd, ...)
github.com/caddyserver/[email protected]/config.go:710 +0x179
github.com/caddyserver/certmagic.(*Config).ObtainCert(0xc000bb85a0, 0x1cee840, 0xc000be2380, 0xc0008c1990, 0x9, 0x0, 0xc000e59cd0, 0x74d9aa)
github.com/caddyserver/[email protected]/config.go:378 +0xd6
github.com/caddyserver/certmagic.(*Config).manageOne.func1(0x1a9a268, 0xc000256800)
github.com/caddyserver/[email protected]/config.go:310 +0x95
github.com/caddyserver/certmagic.(*jobManager).worker(0x2731c20)
github.com/caddyserver/[email protected]/async.go:73 +0x103
created by github.com/caddyserver/certmagic.(*jobManager).Submit
github.com/caddyserver/[email protected]/async.go:50 +0x12d
1.609085889140082e+09 info tls.cache.maintenance stopped background certificate maintenance {"cache": "0xc00018c230"}
2020/12/27 16:18:09.140 INFO tls.cache.maintenance stopped background certificate maintenance {"cache": "0xc00018c230"}
2020/12/27 16:18:09.140 INFO autosaved config {"file": "/home/ice1/.config/caddy/autosave.json"}
ice1@ice2:~/ice1$ 2020/12/27 16:18:09.573 INFO admin stopped previous server
1.60908589648735e+09 debug http.stdlib http: TLS handshake error from 109.76.62.0:58626: no certificate available for 'www.istand.ie'
Here is my config file just as reference.
{
"storage": {
"module": "dynamodb",
"table": "CertMagic",
"aws_endpoint": "",
"aws_region": "eu-west-1",
"aws_disable_ssl": false,
"lock_timeout": 0,
"lock_polling_interval": 0
},
"logging": {
"sink": {
"writer": {
"output": "stdout"
}
},
"logs": {
"": {
"writer": {
"output": "stdout"
},
"encoder": {
"format": "console"
},
"level": "DEBUG"
}
}
},
"apps": {
"http": {
"servers": {
"srv1": {
"automatic_https": {
"disable_redirects": false,
"disable": false,
"skip": [
"IP_HERE"
]
},
"listen": [
":80"
],
"routes": [
{
"match": [
{
"host": [
"IP_HERE"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "OK",
"handler": "static_response"
}
]
}
]
}
]
}
]
},
"srv0": {
"listen": [
":443"
],
"routes": [
{
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"encodings": {
"gzip": {},
"zstd": {}
},
"handler": "encode"
},
{
"body": "Hello, world!",
"handler": "static_response"
}
]
}
]
}
],
"match": [
{
"host": [
"www.icewolf.app"
]
}
],
"terminal": true
},
{
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "static_response",
"headers": {
"Location": [
"https://www.{http.request.host}{http.request.uri}"
]
},
"status_code": 302
}
]
}
]
}
],
"match": [
{
"host": [
"corner.ie",
"icewolf.app"
]
}
],
"terminal": true
},
{
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"encodings": {
"gzip": {},
"zstd": {}
},
"handler": "encode"
},
{
"body": "Hello DNS",
"handler": "static_response"
}
]
}
]
}
],
"match": [
{
"host": [
"*.corner.ie"
]
}
],
"terminal": true
}
]
}
}
},
"tls": {
"automation": {
"policies": [
{
"issuer": {
"challenges": {
"dns": {
"provider": {
"api_token": "<VALID_TOKEN_HERE>",
"name": "cloudflare"
}
}
},
"module": "acme"
},
"subjects": [
"*.corner.ie"
]
}
]
}
}
}
}
VelinGeorgiev
All 7 comments
Thanks for the report!
Can you replicate it without the dynamodb plugin?
This panic should only happen if there is a bug in the storage module:
The panic is on the line with append, which has been correct for... a long time. I suspect the DynamoDB storage plugin is setting IsTerminal: to true for a key that isn't terminal OR the storage structure is malformed somehow.
It would be interesting to see... what is the contents of your database? At least the entry keys.
Pinging @fillup to see if he has any ideas. :+1:
mholt
on 27 Dec 2020
@mholt Thanks for the lightning fast response!
I will give it a try without the module and see if it works.
Here are the contents of the dynamo db as reference. I removed the cert values.
"PrimaryKey (S)","Contents (S)","LastUpdated (S)"
"acme/acme-staging-v02.api.letsencrypt.org-directory/users/[email protected]/name.json","value...","2020-12-27T15:29:52Z"
"acme/acme-staging-v02.api.letsencrypt.org-directory/users/[email protected]/name.key","value...","2020-12-27T15:29:52Z"
"acme/acme-v02.api.letsencrypt.org-directory/users/default/default.json","value...","2020-12-27T13:28:09Z"
"acme/acme-v02.api.letsencrypt.org-directory/users/default/default.key","value...","2020-12-27T13:28:09Z"
"acme/acme-v02.api.letsencrypt.org-directory/users/[email protected]/name.json","value...","2020-12-27T15:28:51Z"
"acme/acme-v02.api.letsencrypt.org-directory/users/[email protected]/name.key","value...","2020-12-27T15:28:51Z"
"certificates/acme-v02.api.letsencrypt.org-directory/corner.ie/corner.ie.crt","value...","2020-12-27T15:30:02Z"
"certificates/acme-v02.api.letsencrypt.org-directory/corner.ie/corner.ie.json","value...","2020-12-27T15:30:02Z"
"certificates/acme-v02.api.letsencrypt.org-directory/corner.ie/corner.ie.key","value...","2020-12-27T15:30:02Z"
"certificates/acme-v02.api.letsencrypt.org-directory/icewolf.app/icewolf.app.crt","value...","2020-12-27T13:28:10Z"
"certificates/acme-v02.api.letsencrypt.org-directory/icewolf.app/icewolf.app.json","value...","2020-12-27T13:28:10Z"
"certificates/acme-v02.api.letsencrypt.org-directory/icewolf.app/icewolf.app.key","value...","2020-12-27T13:28:10Z"
"certificates/acme-v02.api.letsencrypt.org-directory/wildcard_.corner.ie/wildcard_.corner.ie.crt","value...","2020-12-27T15:29:01Z"
"certificates/acme-v02.api.letsencrypt.org-directory/wildcard_.corner.ie/wildcard_.corner.ie.json","value...","2020-12-27T15:29:01Z"
"certificates/acme-v02.api.letsencrypt.org-directory/wildcard_.corner.ie/wildcard_.corner.ie.key","value...","2020-12-27T15:29:01Z"
"certificates/acme-v02.api.letsencrypt.org-directory/www.icewolf.app/www.icewolf.app.crt","value...","2020-12-27T13:28:11Z"
"certificates/acme-v02.api.letsencrypt.org-directory/www.icewolf.app/www.icewolf.app.json","value...","2020-12-27T13:28:11Z"
"certificates/acme-v02.api.letsencrypt.org-directory/www.icewolf.app/www.icewolf.app.key","value...","2020-12-27T13:28:11Z"
"ocsp/corner.ie-125f30b9","value...","2020-12-27T15:30:02Z"
"ocsp/icewolf.app-b2786a27","value...","2020-12-27T13:28:11Z"
"ocsp/wildcard_.corner.ie-8c222ce7","value...","2020-12-27T15:29:01Z"
"ocsp/www.icewolf.app-c11c9d7d","value...","2020-12-27T13:28:11Z"
👍1
There might be two things going on here.
First, IsTerminal should never be true in the users folder; I only discovered that it sometimes was on my file system because Mac would add _files_ (not folders, which are all that is expected in the users folder) to that folder if I merely opened the folder in Finder, Mac's file explorer app. Btw, terminal = true implies file, terminal = false implies folder.
I don't see anything from your DB layout that suggests that there is a terminal key in the users folder. They all have sub-keys. So that looks like one problem, IsTerminal should be false.
The other thing might be that I'm not decrementing i in the loop after splicing out the bad entry (which shouldn't be there in the first place), but when I have time I'll need to double-check how that works with range.
mholt
on 27 Dec 2020
👍1
Ok, I was able to repro a panic by splicing out every element of the array: https://play.golang.org/p/1FPw8iQ8XJB
I can fix this by not using range and by decrementing i after a splice: https://play.golang.org/p/Ensh8QkiGfE
(I knew this and have done this before, I just forgot how it works with range...)
However, we shouldn't even be hitting this case. The keys acme/acme-staging-v02.api.letsencrypt.org-directory/users/[email protected] and acme/acme-v02.api.letsencrypt.org-directory/users/default and acme/acme-v02.api.letsencrypt.org-directory/users/[email protected] are not terminal, as they have sub-keys. So the DynamoDB adapter needs to be returning false for those keys.
mholt
on 27 Dec 2020
👍1
No issues with file system storage. I will switch to it for now. @mholt thanks again for your quick response!
"storage": {
"module": "file_system"
...
},
Cool. I've fixed the panic in CertMagic so I'll close this issue, but the storage implementation will still need to be fixed (consider raising an issue in that repo if you don't hear from any maintainers here).
mholt
on 28 Dec 2020
👍1
Thank you @mholt!
VelinGeorgiev
on 28 Dec 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
mschneider82
路
3Comments
lorddaedra
路
3Comments
wayneashleyberry
路
3Comments
xfzka
路
3Comments
kilpatty
路
3Comments