Caddy: RFE: implement support for systemd's sd_notify

[eliasp]

I'd like to see support for systemd's sd_notify in Caddy.

Advantages of implementing this:

• Caddy can properly signal to systemd once it's actually ready. Currently systemd simply has to assume it is ready to serve requests once the process is started while in reality Caddy might take a little longer than that to establish all listening sockets, deal with ACME, etc.
• Caddy can provide an informational output to systemd's service status, similar to what Apache httpd does (Status: line):

httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled) Active: active (running) since Sat 2016-04-04 8:48:50 UTC; 1 weeks 3 days ago Process: 21400 ExecReload=/usr/sbin/httpd $OPTIONS -k graceful (code=exited, status=0/SUCCESS) Main PID: 390 (httpd) Status: "Total requests: 34; Current requests/sec: 4; Current traffic: 7 B/sec" CGroup: /system.slice/httpd.service ├─ 390 /usr/sbin/httpd -DFOREGROUND ├─15379 /usr/sbin/httpd -DFOREGROUND ├─15858 /usr/sbin/httpd -DFOREGROUND

The basic sd_notify implementation is rather trivial and straight-forward (see references below), but I have no idea where to hook this up within Caddy and how to go about gathering informative data for the status display, so I'm filing this as an RFE for now.

References:

• sd_notify interface/protocol documentation
• Go implementation in coreos/go-systemd
• advanced Go implementation in Merovius/systemd

[wmark]

I am with you on this one and started some work going in that direction:

https://github.com/mholt/caddy/compare/master...wmark:systemd-sockets

Asking /@all: What other _init_ daemons do support fine-grained status notifications? Is it worth changing a TTY title bar, too, if Caddy were the foreground process?

[wmark]

I'd like to classify this as being more important than »nice to have«:

On slow systems Caddy is shown as »ready« but not serving any connections yet. For example, on my arm7 it takes a few seconds to one minute, and I had no idea what was going on until learning that Caddy was indeed loading and parsing certificates.

That said, if the terminal is a TTY then yes, changing the title bar helps a lot.

[weingart]

Please no messing with TTY title bars.

[eliasp]

What about making TTY-manipulation configurable?

[wmark]

I will implement something that accepts plugins, add systemd no it, and we will go from there.

My point-in-time readiness estimation is mid of this week for reviews.

[eliasp]

@wmark while you're at it, it might be a relatively low-hanging fruit to implement support for systemd's watchdog functionality via sd_notify() in Caddy's main loop.

See also:

• https://www.freedesktop.org/software/systemd/man/sd_notify.html#WATCHDOG=1
• https://www.freedesktop.org/software/systemd/man/sd_watchdog_enabled.html#

[wmark]

Sending keep-alive signals in a goroutine will not tell us if any sockets (like, listening sockets) dropped out, if we ran out of file descriptors, or if a thread had been killed by the system.

A proper watchdog implementation would need to cycle over all sockets, ingress and egress, and see if connections get responses.

The _watchdog_ feature is not covered by my PR.

[wmark]

I am closing this in favour of #777. If you have other ideas, please open an issue if and when #777 is in.