Caddy: `root` for static files, `proxy` for the rest (same root)

[eiszfuchs]

1. What version of Caddy are you running (caddy -version)?

Caddy 0.8.2

2. What are you trying to do?

I was trying to finally switch nginx to Caddy on my production server.
Most of the old configurations, I was able to re-write. Except this construct:

try_files $uri @application;
location @application {
    proxy_pass http://application_server;
}

What _should_ happen is that the first try is against static files in the root folder, passing non-static requests to the proxy.

3. What is your entire Caddyfile?

I _thought_ I could just use a root directive to handle static files, followed by a proxy to handle the rest.

localhost:2015

tls off
root web/
proxy / localhost:5000

(If the proxy is commented out, text file content is returned, see below.)

4. How did you run Caddy (give the full command and describe the execution environment)?

Windows, using ./caddy.exe -conf ./Caddyfile

|- web
|  \- text.txt
|- app.py
|- caddy.exe
\- Caddyfile

5. What did you expect to see?

Calling http://localhost:2015/test.txt would give me the content of the text file, calling http://localhost:2015/ would let my application answer.
The request for the existing text file isn't passed to the application at all.

6. What did you see instead (give full error messages and/or log)?

404 on the text file, returned by the application.

127.0.0.1 - - [26/Mar/2016 14:12:28] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [26/Mar/2016 14:12:33] "GET /test.txt HTTP/1.1" 404 -

[eiszfuchs]

Off topic for the curious: For a full switch, I also needed something described in https://github.com/mholt/caddy/issues/167.
That's the full scenario:

root /var/www/project/public;

try_files $uri @application;
location @application {
    proxy_pass http://application_server;
}

location ~ ^/(assets)/ {
    root /var/www/project/public;
}

This case isn't possible with except.
But I guess I'd have wait for https://github.com/mholt/caddy/issues/619 to come to a conclusion, since that's exactly the topic discussed there if I read that correctly.

[abiosoft]

The full scenario is actually easier since assets has its own folder, this should do. May I ask why it is not possible with except? That regex is not complex, the captured group is not used and caddy's default static path matching should be enough for it.

root /var/www/project/public
proxy / http://application_server {
    except /assets
}

For the first scenario, you can do something like this.

rewrite {
    to {path} /proxy/{uri}
}
proxy /proxy http://application_server {
    without /proxy
}

[eiszfuchs]

The reason I'm so unsure about everything at the moment is because both cases are for third-party Ruby applications. I want to turn them off in a few months (or so) anyway, but I need to keep them alive until I notices potential users.

The rewrite trick is pretty neat, thank you. Might have to replace /proxy with something only machines would use, but yeah, that would work - except for my example above, this will return 404 for / and not pass that to my application.

localhost:2015

root web/
rewrite {
    to {path} /proxy/{uri}
}
proxy /proxy localhost:5000 {
    without /proxy
}

[mholt]

What if you change that to line to:

    to {uri} {uri}/ /proxy/{uri}

[abiosoft]

He only wants to try files judging from the nginx config. Also, {path} is preferred for file existence as {uri} may contain query string.

[mholt]

Oh, great point.

    to {path} {path}/ /proxy/{uri}

[eiszfuchs]

Nah, both versions don't work. I suppose because / actually _is_ existent, but I didn't allow browse to happen, so Caddy will 404 that page, since that's what Caddy does in this case.

[abiosoft]

If that's the case, let's add one more condition for base path. Sorry, it gets uglier :expressionless:.

localhost:2015

root web/
rewrite {
    if {path} is /
    to /proxy/{uri}
}
rewrite {
    to {path} /proxy/{uri}
}
proxy /proxy localhost:5000 {
    without /proxy
}

[eiszfuchs]

That works!
I actually _had tried_ this already in my Caddyfile after sending that comment, but when I saw your reply, I figured the order of the rules is kinda important. D'uh.

rewrite {
    if {path} is /
    to /proxy/{uri}
}

Don't worry about the file getting ugly, as I said, these projects are soon gone for good, and even * that* is _much_ cleaner than the original 200-lines long nginx config file for each of them (SSL/TLS with nginx is sort of a pain in the butt).

[abiosoft]

There is another approach you may like. This requires you to have the static files in a subfolder e.g. assets.

localhost:2015

root web
rewrite {
    to /assets/{path} {uri}
}
proxy / localhost:5000 {
    except /assets
}

Yes, the order is very important and that is why the wildcard must be the last. This won't be the case forever, we are looking at a solution.

[eiszfuchs]

If those were projects of mine, I could easy make a static folder (in fact, I do in my projects).
Will keep that in mind and will stay around because I expect changes to Caddyfile syntax anyway since we're pre-1.0. Let's pray for the best when I switch off nginx. :smiley:

Hope others can benefit from this conversation, too.
Anyway, thanks for beeing so supportive! :beers:

[abiosoft]

Glad we found a way out. You're welcome.

I assume it is safe to close this now.

[eiszfuchs]

Yeah, it is!

[corny]

Is there a plan to change the Caddyfile syntax to achieve this in a smarter way?

_Update:_ The example from @abiosoft works for me.

[rindek]

Hello

For anyone interested: the solution given by @abiosoft here: https://github.com/mholt/caddy/issues/695#issuecomment-201879777 works for the version 0.8.x, however for 0.9.x it doesn't.

The actual behaviour is:

• serve any found static content from assets/ directory √
  Example: curl localhost:3000/robots.txt, in logs: 172.17.0.1 - [21/Oct/2016:08:18:09 +0000] "GET /public/robots.txt HTTP/1.1" 200 202
• proxy the request to the upstream with path different than / √
  Example: curl localhost:3000/app, in logs: 172.17.0.1 - [21/Oct/2016:08:15:28 +0000] "GET /app HTTP/1.1" (...) - request proxied properly
• 404 when going to / x
  This is where it doesn't work. For the main path it returns 404, with a log: 172.17.0.1 - [21/Oct/2016:08:15:24 +0000] "GET /assets/ HTTP/1.1" 404 14

I tried to add without /assets in the Caddyfile but it didn't change anything.
The working behaviour for version 0.9.x is possible to achieve using this: https://github.com/mholt/caddy/issues/695#issuecomment-201871982

[mholt]

@rindek Does your Caddyfile use a path in the site address, perchance?

[rindek]

@mholt no, here is the Caddyfile:

localhost:3000

root /app

rewrite {
  to /public/{path} {uri}
}

proxy / rails:3000 {
  except /public
}

gzip
log / stdout

Do not mind the folder name being public it is just a naming change I did. Also an interesting thing here is that in version 0.9.0 it logs out that it is doing a request to / (still returning 404), but from 0.9.1 it logs /public/ (still doing same request to localhost:3000)

[gastlich]

@rindek this worked for me: (rewrite everything to /public, apply proxy except for /static and without /public)

root /home/web/website/

header / -Server

rewrite {
    to /public/{path}
}

proxy /public unix:/home/web/gunicorn.sock {
    except /static
    without /public
    transparent
}


log stdout