Caddy: Using proxy with a query results in an added slash being forwarded

[elvarb]

1. What version of Caddy are you running (caddy -version)?

Caddy 0.9.3

2. What are you trying to do?

I'm trying to make a simple proxy for the http endpoint on NSQ
Any data posted to the site root should be forwarded to the proxy url where the topic is defined

In the NSQ log files I can see that the request goes to the wrong address

3. What is your entire Caddyfile?

localhost:8001
proxy / localhost:4151/pub?topic=caddy

4. How did you run Caddy (give the full command and describe the execution environment)?

Ran the binary, it read the CaddyFile

5. What did you expect to see?

Proxy should forward the connection to the correct destination defined in the config

/pub?topic=caddy

6. What did you see instead (give full error messages and/or log)?

NSQ got the query like this

/pub/?topic=caddy

7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?

If you want to use NSQ then just start NSQd and it will run with the default config. Monitor stdout.

[mholt]

When you say "latest" do you mean latest release or a commit? Please be specific, this project is in motion! For example, we have a recent commit, #1178, that may affect this issue.

[elvarb]

Sorry, version added :) more motion!

That commit very well could solve this

[mholt]

Thanks! Do you have the ability to build from source? Perhaps you would have a chance to check it before I do (I have midterms for another week).

[elvarb]

Not at the moment no, possibly next week.

[tobya]

With my limited testing #1178 4baca884c5e4767b114f3d363188e6570a24e90d does not seem to fix this issue, the extra / is still added to the forward-proxy url

[mholt]

Okay -- I think that was for a different issue (but still could have affected this one). So I guess we need to look into this one more.

[tobya]

I've looked at the code responsible & it seems to assume the proxy will be an upstream directory rather than a file. I will see if I can make progress

[tobya]

I need some discussion on expected behaviour.

Currently what happens is that the proxy is assumed to be a directory which can have first directories and then query parameters added to it.

so given

localhost:8001
proxy / localhost:4151/pub?topic=caddy

visiting localhost:8001 will add the extra / not desired to get localhost:4151/pub/?topic=caddy and I could just prevent this from happening.

However localhost:8001/pages/file.php?other=value will proxy to

localhost:4151/pub/pages/file.php?topic=caddy&other=value

where additional path items are added to the proxy path and additional query parameters are added to the query params.

The proxy works _correctly_ if you use a file ext.

 localhost:8001
 proxy / localhost:4151/pub.php?topic=caddy   

then localhost:8001/pages/file.php?other=value will proxy to

localhost:4151/pub.php/pages/file.php?topic=caddy&other=value

now it is the pub.php file that will be executed by the server and the rest of the path can be retrieved through php environment variables.

So that gives a number of options.

1. Leave it as it is! I am begining to wonder if this is actually a bug, since simply specifying the ext of a php/asp/ etc file solves the issue.
2. Run a check to see if there is a _?_ and if not assume directory, but if there is assume a file and just add parameters. However in this instance what would we do with the information /pages/file.php ?
3. If the request exactly matches the proxy path then don't modify the proxy url, but if the request has additional path or query parameters work as the existing code does.

I'm sure there are other options I havent considered. Any suggestions?

[tobya]

I have created a pull request implementing option 3

[mholt]

@tobya Thanks so much for looking into this. I guess this is complicated; did we decide what we want to do, or how other web servers handle this?

[tobya]

Sorry I havent had time to continue this. The only complexity is deciding what to do. I don't really have access to other webservers at the moment.

If someone else wishes to take this up great, otherwise I'll try to get back to it soon.

From your comment on the PR it seems that the behaviour should stay the same except in the very spcific instance where there is no incoming path we should ensure an extra / is not added to the proxy url.

[mholt]

@tobya Awesome, thanks.

From your comment on the PR it seems that the behaviour should stay the same except in the very spcific instance where there is no incoming path we should ensure an extra / is not added to the proxy url.

... I think that sounds right. Now that we've defined the behavior we want to have, maybe this will be fairly simple.

[tobya]

Just added a pull request #1256 that doesnt fix but is the next step on the way.

[tobya]

My attempts to fix this have introduced unpredictable behaviour with paths. In my opinion

localhost:4151/pub?topic=caddy

and

localhost:4151/pub/?topic=caddy

should point to the same server resource.

[mholt]

Pretty sure this is fixed in v2 (since we rewrote it from scratch).