Caddy: V2: reverse proxy transparency

[sosodev]

1. What would you like to have changed?

In Caddy V1 you could specify transparent with the proxy directive to set the appropriate headers for an app running behind the reverse proxy. In Caddy V2 this seems to be missing meaning you have to set the headers manually, just a little inconvenient.

2. Why is this feature a useful, necessary, and/or important addition to this project?

It makes running an app behind a reverse proxy with Caddy, something that's already pretty easy, even easier! Most apps respond to the forwarding headers correctly and that means a lot of reduced headache for devs using Caddy V2.

3. What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

As mentioned before you can just add the headers yourself like

reverse_proxy {
    header_up X-Forwarded-Proto https
    etc
}

so it's not critical by any means just some QOL

[mholt]

Hm, yeah, I guess this would be nice to have huh.

In v1, the transparent shortcut does the following:

header_upstream Host {host}
header_upstream X-Real-IP {remote}
header_upstream X-Forwarded-For {remote}
header_upstream X-Forwarded-Port {server_port}
header_upstream X-Forwarded-Proto {scheme}

Do you like what behavior it hides from you?

[Immortalin]

@mholt random question, does the reverse-proxy command in Caddy V2 include WebSockets?

[pascalgn]

I thought about this feature when implementing #2801 but in the end decided against it. IMO it's better to specify the headers manually. It would be a good thing to add this to the documentation, though, especially to help people who are coming from v1

[mholt]

@Immortalin Yes websockets work without needing any extra config, because all headers from downstream are copied into the upstream.

[Immortalin]

@mholt thanks!

[atonse]

I'm trying to figure this out at the moment setting up Caddy 2 and all I'm finding that the original, elegant configuration of Caddy seems to be replaced with this unnecessary new config system that just looks no different from apache (at this point nginx is more elegant, and it was always more elegant than apache). Is there really any benefit to all this new config? What happened to the elegant one-liners? That was the whole appeal of Caddy, curated and well-thought-out defaults.

The documentation only shows JSON but it is very hard for those of us who want the terse and very simple best-practice based config and don't want to define everything.

I've got this:

mysite.com {
  reverse_proxy * {
    to localhost:3000
  }
}

But that's not really working and is giving me a 502 gateway error even though I can easily access localhost:3000 using curl.

Update: I got this working by restarting instead of reloading. I'm not sure reload is working properly. The config above is what was not getting picked up in the reload. Should I create a separate Issue?

[Immortalin]

TOML would be a good choice of a config language

[mholt]

@atonse

I'm trying to figure this out at the moment setting up Caddy 2 and all I'm finding that the original, elegant configuration of Caddy seems to be replaced with this unnecessary new config system that just looks no different from apache (at this point nginx is more elegant, and it was always more elegant than apache). Is there really any benefit to all this new config? What happened to the elegant one-liners? That was the whole appeal of Caddy, curated and well-thought-out defaults.

The documentation only shows JSON but it is very hard for those of us who want the terse and very simple best-practice based config and don't want to define everything.

Please be patient as we build new documentation for v2. What you are seeing is not the final product. We are still in beta, remember. That said, I am glad you are trying Caddy 2 -- thank you for using it while it's still in beta.

I can assure you that Caddy 2 will preserve the same easy-to-use-ness and good defaults as Caddy 1, but with many improvements and enhancements.

You should be able to simplify your config to:

mysite.com
reverse_proxy localhost:3000

which is simpler than in Caddy 1.

You don't even need a config file for this:

$ caddy reverse-proxy --from mysite.com --to localhost:3000

(I think, I haven't tested this CLI command as much yet. But you might try it out)

@Immortalin

TOML would be a good choice of a config language

Feel free to contribute a config adapter! Just like @iamd3vil did here for YAML: https://github.com/caddyserver/caddy/issues/2876#issuecomment-552824668 -- it'd be like 20 lines of code.

[atonse]

Thank you @mholt – yes it's good to remind me that it's a beta. :-)

I agree that the main issue is probably more on the doc side, since it focuses on JSON configuration.

Regarding not needing a config file, I'm eventually going to run about 9 different sites (as 9 docker containers) as reverse proxies and I didn't really want to run 9 copies of Caddy (and 9 systemd units), which is why I didn't go the command line route.

[aeroxy]

How can we achieve header_upstream Host {host} in V2, this doesn't seem to work:

caddy.com:443 {
  tls /etc/certs/caddy.crt /etc/certs/caddy.key
  reverse_proxy /caddy 127.0.0.1:8443 {
    header_up Host {host}
    transport http {
      tls
    }
  }
}

[pascalgn]

Possibly try

    header_up Host {http.request.hostport}

[aeroxy]

Possibly try

    header_up Host {http.request.hostport}

hostport sounds like, hostport

Did you mean http.request.host?

[pascalgn]

I think in most cases both will work, but to be completely safe, Host should contain hostname + port: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Host ({http.request.hostport} will resolve to e.g. example.com:8080)

[DoTheEvo]

yes, please add transparent proxy keyword... I am writing some basic how-to caddy with examples, and bam suddenly nice readable config has shitload of boilerplate code if there is desire to forward stuff, which mostly is.

[mholt]

Done

[mholt]

Actually, I was too hasty.

"Transparent mode" is the default in v2 already: the Host header will go to the upstream unmodified. X-Forwarded-For is also added for you. Other headers like X-Real-Ip and X-Forwarded-Proto can be added manually... for now, that's how it is anyway.

The idea is that, by default, Caddy 2 passes headers thru, allowing you to more easily make any changes without first having to undo Caddy's hidden changes.

I'm going to revert that commit, since transparent mode is default already, so the transparent keyword is mostly useless. If you need the other headers, you can just add them yourself. But not all apps need those.

If you want the upstream Host header to be the actual upstream's host, you can do:

header_up Host {http.reverse_proxy.upstream.hostport}

Which is more like the v1 proxy's default.

[athei]

It is nice and all that this headers are automatically added. But there seems no way to remove them. This for example does not work:

reverse_proxy http://127.0.0.1:8989 {
    header_up -X-Forwarded-For
    header_up -X-Forwarded-Proto
}

[francislavoie]

@athei Thanks for pointing that out, could you please open a new issue for this?

[athei]

@athei Thanks for pointing that out, could you please open a new issue for this?

https://github.com/caddyserver/caddy/issues/3584

[NightMachinary]

Actually, I was too hasty.

"Transparent mode" is the default in v2 already: the Host header will go to the upstream unmodified. X-Forwarded-For is also added for you. Other headers like X-Real-Ip and X-Forwarded-Proto can be added manually... for now, that's how it is anyway.

The idea is that, by default, Caddy 2 passes headers thru, allowing you to more easily make any changes without first having to undo Caddy's hidden changes.

I'm going to revert that commit, since transparent mode is default already, so the transparent keyword is mostly useless. If you need the other headers, you can just add them yourself. But not all apps need those.

If you want the upstream Host header to be the actual upstream's host, you can do:

header_up Host {http.reverse_proxy.upstream.hostport}

Which is more like the v1 proxy's default.

@mholt Is X-Forwarded-For set to the HTTP request’s originating IP or does it use the same header of the request? I’m asking because I am relying on the X-Forwarded-For to show the real IP of the request in my backend, and the original request’s X-Forwarded-For can be easily spoofed by the requester.

[francislavoie]

Both. Caddy appends the originating IP to the header if it already exists.

https://github.com/caddyserver/caddy/blob/e9b1d7d/modules/caddyhttp/reverseproxy/reverseproxy.go#L472-L480

For next time, please ask your questions on the forums instead of on old closed issues. https://caddy.community