Caddy: Support for TLS session resumption in "fleet-mode"

[costela]

1. What version of Caddy are you using (caddy -version)?

0.11.0

2. What are you trying to do?

caddytls.standaloneTLSTicketKeyRotation rotates keys on a per-node basis, so if we have multiple Caddy instances, each one gets different ticket keys and clients are then unable to resume TLS sessions from other instances.

It would be great if Caddy would support TLS session resumption also in "fleet-mode", so that we don't have to chose between high-availability+load-balancing and client connection establishment speed.

The naming of standaloneTLSTicketKeyRotation seems to indicate someone already thought about the necessity of a "non-standalone" version, but I couldn't find another issue to track this. Did I maybe overlook something?
Also, this is probably part of a bigger effort to make Caddy more "cluster-aware". As I see it, it would probably require either some central KV (like #1918), or awareness of other nodes in fleet/cluster to coordinate session ticket key generation.

Just as an initial suggestion: at work we implemented a minimalistic reverse-proxy that (ab)uses groupcache to force the session ticket generation to happen in one peer, and spread it around to the other peers on demand. This is a simplistic approach, and - as stated above - requires that nodes know who are their peers, but something like this could be a starting point. Thoughts?

[costela]

Just for the record, there is another simpler (but flawed) approach: generating the session key based on time (truncated to the magnitude of TicketRotateInterval) + some pre-shared key + the tls.Config cert (assuming fleet-mode certificate sharing is working).
This would have the advantage of working without any peer-coordination, but has the serious drawback of _sacrificing PFS_: leaking the pre-shared key would mean any previously recorded encrypted connections could be decrypted. This kind of defeats the whole point of having session ticket rotation.

[wmark]

Aye! I am the original author of said functionality. And you guessed right.

Historically, the biggest challenge with individual "secrets" for session resumption, stored server-side, has been a secure distribution and storage. That is, if an attacker got the server-side cache or intercepts distribution (or fetching, which would be the passive counterpart), then all previous session could be decrypted. That's why session tickets have been introduced, and storage is client-side.

With this history in mind, any distributedTLSTicketKeyRotation shall not have any predictable or static component. Other than timing, perhaps. Specifically, no PSK and not using any part of any certificate or secret key. Use randomness, and you already have a TLS setup to distribute it node-to-node.

Omitting the steps leading up to this: A new key N will be inserted into the set of legacy keys Oooo in two stages: [ONoo] → [NOoo]

As long as eventually there's one N, any previous imposters N_i do no harm (other than being considered for decryption while everything is in an ephemeral state).

And yes, all what's now missing is a way of triggering key rotation (hint: probabilistic) and leader election (for authority over selecting the N out of any ). I've already explained in Caddy's old chat room, you neither need any new age algorithms to accomplish that, nor any shared components (or even time).

[wmark]

By the way, you can click on "blame" (or git blame…) to spot who authored what (if git has been used as intended all the time) to get in touch with the author directly.

[costela]

@wmark thanks for the thorough answer! Sorry for letting it unack'd for so long.

@mholt thanks a lot for the feedback! I had just started to look into this again yesterday and was going through the cluster plugin docs trying to find out if it was possible to implement it there. Guess you unwittingly answered my question! :)
Is there any ongoing concrete work, or is this more like "we have a design, but would accept PRs"?

[mholt]

Caddy 2 dev is currently private, but we'll open it up to the public later this summer probably. If you submit a PR for Caddy 1, we could potentially reuse it in Caddy 2.

[mholt]

I have finished implementing this into Caddy 2. @costela, I will reach out to you about giving it a try.

[may573]

I have finished implementing this into Caddy 2. @costela, I will reach out to you about giving it a try.

Hi mholt, could you please guide me on how to enable this feature on coreDNS which is based on caddy? We are suffering from the question how to enable tls session resumption on our coreDNS clusters.

[mholt]

@may573 You should ask CoreDNS. But CoreDNS doesn't use Caddy 2, so it doesn't support this feature. You'll have to ask them to support it.

[may573]

@mholt OK, thanks a lot for your suggestion, I'll try to ask CoreDNS developers for this request. Not sure whether they have planned this upgrading.

[may573]

@mholt I have another question, do caddy support 0-rtt in tls1.3? If not, is there a plan for it?

[costela]

@may573 please refrain from off-topic questions on issues. This needlessly spams subscribers.
Please consider asking in https://caddy.community instead.

[may573]

@costela Sorry for disturbing, I'll ask in forum for off-topic questions～