Caddy: Using environment variables to configure TLS

[francislavoie]

1. What version of Caddy are you running (caddy -version)?

0.9.3

2. What are you trying to do?

I want to use a .env file as the sole file to configure to simplify things for end-users that use my on-prem docker deployed solution which uses Caddy as a server and proxy.

3. What is your entire Caddyfile?

https://{$CADDY_WEB_HOST} {
    root /var/www/html/web
    fastcgi / php-fpm:9000 php 
    rewrite {
        to {path} {path}/ /index.php?{query}
    }
    gzip
    log /var/log/caddy/access.log
    errors /var/log/caddy/error.log

    tls {$CADDY_TLS_PARAM}
}

Along with this, this is the .env file template I have users fill in, i.e. uncomment one of the CADDY_TLS_PARAM variables to enable SSL.

#-----------------#
#----- Caddy -----#
#-----------------#

# The host name of the application
CADDY_WEB_HOST=yoursite.example.com

# Uncomment one of the TLS directives below to enable TLS
#   Explanation for each directive in the order they appear
#   1. Enables usage of a certificate and key signed by a CA.
#      This certificate file should be a bundle: a concatenation 
#      of the server's certificate followed by the CA's certificate
#   2. Enables automatic HTTPS from letsencrypt. The email is for
#      registration with letsencrypt.
#   3. Enables automatic generation of a self_signed certificate.
#      This is only recommended for debugging or testing purposes.
#      Will not work when attempting to register users.
#   4. Disables TLS entirely and uses HTTP only. Will not work when
#      attempting to register users, useful in development.

#[email protected]
#CADDY_TLS_PARAM="/srv/ssl/yoursite.crt /srv/ssl/yoursite.key"
#CADDY_TLS_PARAM=self_signed
CADDY_TLS_PARAM=off

4. How did you run Caddy (give the full command and describe the execution environment)?

With docker

5. What did you expect to see?

I expect a string with a space to be parsed as the cert and key parameters but it instead gets parsed as an email for letsencrypt.

6. What did you see instead (give full error messages and/or log)?

The TLS command gives me a "not a valid e-mail address" error when booting up.

7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?

Set environment variables as above, like CADDY_TLS_PARAM="/srv/ssl/yoursite.crt /srv/ssl/yoursite.key" for example with a valid cert and key.

Is there something I'm missing? How could I make this work? I could have the users modify the Caddyfile, but I prefer to abstract that away from them as much as possible, so I can use shell scripts to modify the .env file easily.

[francislavoie]

Looking into the source code, it looks like the issue arises with this bit of code: https://github.com/mholt/caddy/blob/53e117802fedd5915eeb32907873d8786a4b2936/caddytls/setup.go#L40

Essentially it's trying to tokenize the arguments but I'm passing it as a string with a space in between and that's not recognized as separate tokens. Where does the environment variable expansion happen? How does it handle quoted strings? Is it just writing in the string as-is?

I'd really like this issue resolved so I don't need to keep manually editing the Caddyfile in deployments.

[mholt]

Indeed you're right, an env variable is replaced on a per-token basis; in other words, the value of an env var can't assume the value of multiple tokens. We do a local replacement in the moment, not a global search-and-replace before parsing begins. The replacement happens in parse.go.

I guess we need to decide if this is something we want to keep or change, as you'd like.

I wonder if a global replacement scheme (i.e. evaluating env vars and doing the replacement when loading the input files) would break any tests?

Even if it doesn't break anything, do we really want to allow entire Caddyfiles to be specified in an environment variable? Do we recursively search-and-replace? I mean, this would be weird:

{$SITE_ADDR} {$OPEN_CURLY_BRACE}
    {$DIRECTIVES}
{$CLOSE_CURLY_BRACE}

Right now, the localized replacement prevents you from using env vars to specify syntax and punctuation... it limits you to putting only very specific values in them, which I kind of like. Would you want the above syntax to be possible? (It's dumb, I know, but you know people are going to do it at some point if we let it happen.)

(FWIW, I vote we don't change it.)

[francislavoie]

Would it be possible to split on spaces in an environment variable, maybe just specifically for the tls directive? It seems weird that it's impossible to fully configure TLS with environment variables. That seems to me like a very likely use-case for environment variable expansion.

I do understand your point though, definitely doesn't make sense to allow directives to be configured using environment variables, quite dangerous.

Thanks for reading

[mholt]

I am definitely not going to code in exceptions for certain directives, sorry. I think it's misleading to have one env variable in the Caddyfile that actually results in 2 values/tokens.

One thing you could do is have two files, with tls <one arg> in one and tls <two> <args> in another, then use an env var to specify import {$WHICH_TLS} file to import, for example.

So, thanks again for the issue, but I don't think we should change this. :)

[francislavoie]

Oh okay, neat. I've been browsing through the Caddyfile docs for the past few months and I have no idea how I missed the import directive. That definitely helps. Thanks for pointing that out, that should work for me.

I'd like to mention that it might be worth documenting this limitation for env vars somewhere (unless I missed it?)

[mholt]

Import is probably the best you can do for what you're trying to do. It's not super great, but it should work.

And this limitation is not explicitly documented, you're right. Right now it says env vars can be used as "addresses and arguments" but it doesn't clarify that an environment variable does not expand out to multiple arguments. I've updated the docs locally so they'll go out with the next deploy.

[francislavoie]

I'm only getting around to implementing this now, and I guess I'll find out when I try it in a prod environment, but do environment variables get parsed in imported files? Edit: nevermind, after reading the import docs I think how it'll work is that the import gets parsed before doing anything else, then env after.

For full feature coverage I'm aiming to set it up like this:

caddy_tls_auto:

tls {$CADDY_TLS_EMAIL}

caddy_tls_custom:

tls {$CADDY_TLS_CERT} {$CADDY_TLS_KEY}

caddy_tls_selfsigned:

tls self_signed

caddy_tls_off:

tls off

And loaded from the Caddyfile like this:

import /srv/sslmode/{$CADDY_TLS_MODE}

And it would be configured like this:

# 1. Automatic HTTPS from letsencrypt
CADDY_TLS_MODE=caddy_tls_auto
[email protected]

# 2. Custom certificate and key
# CADDY_TLS_MODE=caddy_tls_custom
# CADDY_TLS_CERT=/src/ssl/cert.pem
# CADDY_TLS_KEY=/src/ssl/key.pem

# 3. Self signed certificate (FOR DEBUGGING)
# CADDY_TLS_MODE=caddy_tls_selfsigned

# 4. Disable TLS entirely
# CADDY_TLS_MODE=caddy_tls_off

[naviat]

@francislavoie sorry for open this but what's happened with your solution in prod

[fbartels]

@naviat not op, but since I had a similar requirement I just implemented the example given. it works just as shown above.

Just be aware that the functionality provided by caddy_tls_selfsigned and caddy_tls_off is practically duplicated. You can achieve the same by defining CADDY_TLS_EMAIL as self_signed or off.

[francislavoie]

Sorry I missed this one

what's happened with your solution in prod

That's a bit of a vague question. What do you mean? It worked perfectly fine.

Just be aware that the functionality provided by caddy_tls_selfsigned and caddy_tls_off is practically duplicated. You can achieve the same by defining CADDY_TLS_EMAIL as self_signed or off.

True, but I was aiming to have as simple UX as possible since this was meant to be configured via a bash config wizard by non-technical people. The wizard would comment out every mode, then find the appropriate env line for the option chosen then uncomment it.

I've since moved on from that project and company, so I don't have that requirement anymore. It worked perfectly well though.