Caddy: Caddy is adding trailing slash to reverse proxy requests

Created on 14 Sep 2017 · 5Comments · Source: caddyserver/caddy

Caddy is adding a trailing slash to a reverse proxy request. IMO, a trailing slash should only be passed upstream if it is part of the configuration (i.e. proxy / 10.0.7.21:8080/sub-dir/)

1. What version of Caddy are you using (`caddy -version`)?

Caddy 0.10.8

2. What are you trying to do?

Reverse proxying

3. What is your entire Caddyfile?

sub.domain.com {
  proxy / 10.0.7.21:8080/sub-dir
}

4. How did you run Caddy (give the full command and describe the execution environment)?

/usr/bin/caddy --conf /etc/Caddyfile --log stdout

5. Please paste any relevant HTTP request(s) here.

http://sub.domain.com

6. What did you expect to see?

Content of 10.0.7.21:8080/sub-dir

7. What did you see instead (give full error messages and/or log)?

Content of 10.0.7.21:8080/sub-dir/ (with a trailing slash, causing a 404 error in this case)

Source

sphr2k

Most helpful comment

This is working as intended. Look:

By the spec, the resource being requested is never empty. So if you make a request to https://example.com, the client itself will make the request for https://example.com/. You can see this behavior with curl -v https://example.com

> GET / HTTP/2
> Host: example.com
> User-Agent: curl/7.53.1
> Accept: */*

So it does make sense for caddy to proxy it to sub-dir/, this is what it is told to do. You could however argue that would be good if caddy sees requests to / and treat them as if they where for na empty resource. But then, it would be odd that https://sub.domain.com/ goes to one place and add a single letter https://sub.domain.com/a sends a request with two additional chars.

For most places it is responsibility of the backend to treat requests ending with a slash as requests without it. In fact if you make a request for a static file such as https://sub.domain.com/file.txt/, it will be treated as equal to https://sub.domain.com/file.txt. The best approach here, IMHO, is for the backend of the proxy to treat https://sub.domain.com/sub-dir/ as the same as https://sub.domain.com/sub-dir.

lbguilherme on 18 Sep 2017

👍2

All 5 comments

Thank you for the issue. Is this happening if you do a curl request also? What response do you get to

curl https://sub.domain.com

tobya on 16 Sep 2017

Hi,
thank you for taking the issue. Here is the output to curl https://sub.domain.com:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
    <head>
        <title>Error: 404 Not Found</title>
        <style type="text/css">
          html {background-color: #eee; font-family: sans;}
          body {background-color: #fff; border: 1px solid #ddd;
                padding: 15px; margin: 15px;}
          pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
        </style>
    </head>
    <body>
        <h1>Error: 404 Not Found</h1>
        <p>Sorry, the requested URL <tt>&#039;https://sub.domain.com/sub-dir/&#039;</tt>
           caused an error:</p>
        <pre>Not found: &#039;/youtube-dl/&#039;</pre>
    </body>
</html>

This would, of course, work, if the target app being proxied were able to handle the trailing slash, but in this case, it isn't. Since the trailing slash is absent in the config as well as in the request, caddy shouldn't add it (in my opinion - maybe there are other aspects to this that I'm not aware of).

sphr2k on 18 Sep 2017

This is working as intended. Look:

> GET / HTTP/2
> Host: example.com
> User-Agent: curl/7.53.1
> Accept: */*

lbguilherme on 18 Sep 2017

👍2

I agree with Guilherme, I think it's the responsibility of the program processing the request to figure out whether the final slash is valid or not and then adjust accordingly. As he said, Caddy will redirect clients to the proper, canonical URL when accessing static files.

So in this case, you'd have to make an HTTP request without your HTTP client adding the trailing slash. So, arguably, the error here is not with Caddy, but is with the HTTP client or the backend in this case.

RFC 3986:

If a URI contains an authority component, then the path component
must either be empty or begin with a slash ("/") character.
...
A path is always defined for a URI, though the
defined path may be empty (zero length).

So uh, I'm not exactly sure what to make of this yet. I feel like we've encountered this before and decided to have it be this way for a reason but I can't recall where (or for sure, if we did) yet...