Caddy: Proposal: Caddyfile directives for grouping per-server options

[francislavoie]

We have plenty of per-server options available in the JSON that don't yet have Caddyfile equivalents. See here: https://caddyserver.com/docs/json/apps/http/

After discussing with @whitestrake, I think we can group the ones that are missing into 3 new directives. Proposed syntax below:

timeouts {
    read_body   <duration>
    read_header <duration>
    write       <duration>
    idle        <duration>
}

limits {
    max_body_size   <size>
    max_header_size <size>
}

protocol {
    allow_h2c
    experimental_http3
    strict_sni_host
}

Notice that max_body_size is not one of the server options in JSON. This would map to the request_body handler which is currently only available in JSON, with the concession that from the Caddyfile, no matchers could be used (i.e. matching all requests), and it would be added at the top of the routes.

Every option other than max_body_size would just be directly passed through to the JSON as-is.

Since experimental_http3 is already a global option, the global option would of course take precedence in enabling it for all servers, and this new per-server option wouldn't offer an opt-out. Either on globally, or on per-server.

[mholt]

We did something like this in Caddy 1 but it was not a clean mapping, and can never be, as long as more than 1 site is served on a listener. The reason being that all these settings apply to the HTTP _servers_, not individual _HTTP requests_ except for, as you note, max_body_size, which is a middleware-enforced thing -- and I don't think it's a good idea to combine those because the semantics are so different.

This will need more consideration before any implementation is done. (And is another reason why I don't really love the Caddyfile, esp. for advanced use cases.)

[AtjonTV]

I would love to see max_body_size find its way into the Caddyfile. I dislike the idea of writing a lot of JSON for my Webserver just as I dislike writing a lot of XML (... Apache ...). This is why I event started to use Caddy as a primary Server.

As of right now I have 302 formatted lines of Caddyfile that transform into 1997 lines of formatted JSON when fead into Caddy.

There is NO WAY I am going to touch that 2k mess of JSON by hand :roll_eyes:

I would rather go in and hardcode that setting for the one domain I need it for, if there is no Caddyfile directive xD

[francislavoie]

New proposal. Global option instead, with a syntax like this:

{
    servers [<listener_matcher>] {
        listener_wrappers {
            <TBD...>
        }
        timeouts {
            read_body   <duration>
            read_header <duration>
            write       <duration>
            idle        <duration>
        }
        max_header_size <size>
        protocol {
            allow_h2c
            experimental_http3
            strict_sni_host
        }
    }
}

So, you have an optional "listener matcher" which is essentially a comparison against the listen address of each server in the underlying JSON; if it matches, then this config applies to that server. If no listener matcher is set, then it applies to all servers.

This roughly aligns with the existing request matchers concept we have, except this would just apply once, right near the end of the Caddyfile adaption. If you define more than one server options block, then the most specific one will apply, meaning you could have one that applies to all, and one that just applies to :443, for example.

[mholt]

That's better, but I'm worried about the complexity it inherently adds to the Caddyfile adapter -- also, we currently have global options that cover several of these (perhaps less-than-optimally, since it's not per-server), so we'd either have to deprecate them or override them or something (gets confusing).

[francislavoie]

so we'd either have to deprecate them or override them or something

I think we have precedence for this with the tls directive and moving some options to issuer, for example (and erroring out if both are used).

[francislavoie]

I have a WIP implementation in #3836 😄

[francislavoie]

PR is all done 🎉

[segevfiner]

It is rather common to have to set a max body size for most of an API, but having to change it for some of the paths that handle, for example, file uploads.

This looks like it won't provide a way to achieve that.

[francislavoie]

@segevfiner max body size is already merged in https://github.com/caddyserver/caddy/pull/3859. It's not a server option, it's a request handler.