Caddy: Encoded slash in URL

[flecno]

I want to address a resource with the name /. So rabbitmq api requests look like this: /api/queues/%2F/queue.fetchtasks. I can't proxy this requests with caddy, because caddy modify the url path.

My simple Caddyfile:

localhost:8888 {
  # proxy to "nc -l localhost 9999"
  proxy /test http://localhost:9999
}

If I run curl localhost:8888/test/%2F/foo, then caddy decodes the path before proxy (%2F to /). This results in /test///foo but it should be /test/%2F/foo. For apache there is an option AllowEncodedSlashes. Is there any equivalent in caddy?

[mholt]

Hmm, good question. Never even heard of that before. :smile: Anyone want to look into this?

[jleedev]

I would argue that a front-end HTTP proxy should _never_ do this kind of unescaping. It breaks composability.

Bearing in mind that I don’t have extensive experience coupling various applications to various load balancers, it nevertheless seems that this unescaping behavior is a vestige of CGI, where the PATH_INFO variable is decoded from its URL representation to look like a usable filesystem path.

Modern applications aren’t CGI anymore. Their interface is HTTP, and so should be equally usable pointing the client directly at it for development and testing, and proxying the requests for production. Likewise, a request traversing two reverse proxies will have the URL path double-decoded. Who could possibly be relying on this multitude of behaviors?

I would modify caddy to make as few changes as possible to proxied requests, and only when necessary to comply with RFCs and common practice. (Is there even an RFC for reverse proxies?)

[jupiter]

I am unable to reproduce this with the provided config.

For $ curl localhost:8888/test/%2F/foo, I get:

$ nc -l localhost 9999
GET /test/%2F/foo HTTP/1.1
Host: localhost:9999
User-Agent: curl/7.47.0
Accept: */*
X-Forwarded-For: ::1
Accept-Encoding: gzip

@flecno Which version of Caddy are you running and on what platform?

[flecno]

Oh year.. I can reproduce this with version 0.8.1 (darwin amd64). I see there is version 0.8.2 and this version has fixed the problem! :+1:

So thanks for your help.

[mholt]

Huh, cool, I'm not sure why! But glad it's better now.

[mholt]

A little birdie told me this might be because of a change to http.Dir in Go 1.6, and 0.8.2 is the first to be built on Go 1.6. (Something about calling path.Clean.)

[robxu9]

Unfortunately bumping an old issue, but I'm on Caddy 0.8.3 and it seems that I'm still running into this problem. Jenkins attempts to detect whether a reverse proxy is broken, and so hits a url like http://192.168.0.108/jenkins/administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/testForReverseProxySetup/http%3A%2F%2F192.168.0.108%2Fjenkins%2Fmanage/.

However, because Caddy seems to decode the second %2F% and changes it to an actual /, Jenkins only sees the http:/ part of the URL, not the whole http://192.168.0.108/jenkins/manage.

[jupiter]

I can reproduce this. Currently looking into it.

[jupiter]

@wmark I see this was introduced here https://github.com/mholt/caddy/commit/f31875dfdeb73db9dddf6a2a175a6b0b349d9b28

We should probably send the original path on the upstream servers. I see that you noted that we can use URL.RawPath in middlewares, but modifying the incoming request path at this point may not be expected by all middlewares. In fact, the recommendation in the Go docs is to use the URL.EscapedPath method and to not use URL.RawPath directly, which is exactly what is happening in the upstream request.

We have two options:

1. Remove this modification of the URL.Path and ensure the integrity of the request URL for all middlewares, or,
2. Reconstruct a URL in the Proxy middleware only.

What do you think?

[wmark]

If middleware _proxy_ needs the verbatim path, then don't pass the cleaned-up one.

[mholt]

@jupiter Does URL.EscapedPath not work for this case? I wonder why it suggests not reading RawPath directly.

[jupiter]

@mholt It looks like URL.EscapedPath creates a new escaping of URL.Path if a URL.RawPath is not available or invalid.

@wmark The above PR fixes this particular issue. However, we should be careful not to introduce potential latent defects wherever we are matching against a modified URL.Path, such as in middleware/proxy/proxy.go#L81, middleware/redirect/redirect.go#L22, middleware/headers/headers.go#L25, etc.

[wmark]

@jupiter I am the wrong person to talk to about middleware _proxy_, I am afraid. The way I see it your changes do not, and will most probably not affect the core of Caddy. Feel free to ping me if that changes or when you have direct questions. Until then I unsubscribe from this issue because I cannot contribute anything to it at the moment.

[Cinerar]

Reproduceble with (got same problem with rabbit as in first post)
Caddy 0.9.5 (+7d15435 Sun Apr 16 00:21:33 UTC 2017)

localhost:8888 {
  # proxy to "nc -l localhost 9999"
  proxy /test http://localhost:9999
}

 curl localhost:8888/test/%2F/foo



md5-c3abc3fac358d5a234961ec1459f2591



nc -l localhost 9999
GET /test/foo HTTP/1.1
Host: localhost:9999
User-Agent: curl/7.47.0
Accept: */*
X-Forwarded-For: 127.0.0.1
Accept-Encoding: gzip