Caddy: v2: Any requested document returns always 0 bytes

[wille-io]

If I request any document (doesn't matter if it exists or not), Caddy returns 200 and a 0 byte document.
I hope I'm doing something fundamentally wrong here.

Version used is: caddy_2.0.0-rc.1_Linux_x86_64.tar.gz
OS: Ubuntu 18.04.4 (Also tested with Fedora 31 - same result)

Caddyfile:

{
  debug
}

:8000

log
root * /home/wille/test/

Directory /home/wille/test/ contains an index.html with content (a pure text file):

Test123!

Starting caddy with caddy run -config Caddyfile returns:

2020/04/04 20:07:11.283 INFO    using provided configuration    {"config_file": "Caddyfile", "config_adapter": ""}
2020/04/04 20:07:11.284 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
2020/04/04 20:07:11.284 INFO    tls cleaned up storage units
2020/04/04 20:07:11.285 DEBUG   http    starting server loop    {"address": "[::]:8000", "http3": false, "tls": false}
2020/04/04 20:07:11 [INFO][cache:0xc000685a40] Started certificate maintenance routine
2020/04/04 20:07:11.285 INFO    autosaved config    {"file": "/home/wille/.config/caddy/autosave.json"}
2020/04/04 20:07:11.285 INFO    serving initial configuration

A simple wget 127.0.0.1:8000 returns an empty index.html and caddy logs:

INFO    http.log.access handled request {"request": {"method": "GET", "uri": "/", "proto": "HTTP/1.1", "remote_addr": "127.0.0.1:38206", "host": "127.0.0.1:8000", "headers": {"Accept": ["*/*"], "Accept-Encoding": ["identity"], "Connection": ["Keep-Alive"], "User-Agent": ["Wget/1.19.4 (linux-gnu)"]}}, "common_log": "127.0.0.1 - - [04/Apr/2020:20:07:14 +0000] \"GET / HTTP/1.1\" 0 0", "latency": 0.000007202, "size": 0, "status": 0, "resp_headers": {"Server": ["Caddy"]}}

Requesting a non existing file with wget 127.0.0.1:8000/filedoesnotexist returns exactly the same:

INFO    http.log.access handled request {"request": {"method": "GET", "uri": "/filedoesnotexist", "proto": "HTTP/1.1", "remote_addr": "127.0.0.1:38222", "host": "127.0.0.1:8000", "headers": {"Accept-Encoding": ["identity"], "Connection": ["Keep-Alive"], "User-Agent": ["Wget/1.19.4 (linux-gnu)"], "Accept": ["*/*"]}}, "common_log": "127.0.0.1 - - [04/Apr/2020:20:15:47 +0000] \"GET /filedoesnotexist HTTP/1.1\" 0 0", "latency": 0.000010255, "size": 0, "status": 0, "resp_headers": {"Server": ["Caddy"]}}

[mholt]

You forgot to enable the static file server :)

https://caddyserver.com/docs/caddyfile/directives/file_server

This is also mentioned in the upgrade guide here, if you're coming from v1: https://caddyserver.com/docs/v2-upgrade#primary-changes

Hope that helps!

[wille-io]

Ah I see! But why does Caddy always return 200 with 0 bytes for unrouted paths?

[mholt]

Ah, well, that's just how an empty response is written. The semantics are a little nuanced. Basically, it's not that a requested resource wasn't found (404), per-se, it's just that the server wasn't configured to write a non-empty response. 200 means it is working and responded as configured, but empty because it wasn't configured to fill in the response.

[matthewbloch]

Just found this discussion, so I solved my problem thankyou 😊 but it was confusing not to get a 404 when I'd not defined a route properly.

The linked page says "Most often, the file_server directive is paired with the root directive to set file root for the whole site." - but do you have an example of when the root directive makes sense _without_ a file server?

[mholt]

@matthewbloch

but do you have an example of when the root directive makes sense without a file server?

Yes, anything that uses {http.vars.root} (presumably to access the file system) -- for example, the php_fastcgi directive or the webdav plugin or any other configuration that uses that placeholder, either implicitly or explicitly. It just sets a variable, and there's more uses for it than just serving static files.

The root directive docs even say this:

This directive does not automatically enable serving static files, so it is often used in conjunction with the file_server directive or the php_fastcgi directive.

That variable is also used by the templates handler which can operate on static files or anything that was reverse-proxied as well.

(edit: wow, github butchered my post)

[mbernasocchi]

I just spent a good half hour on this, so almost a personal reminder. a wrongly configured site address will do the same.
I had localhost:8080 but was calling the server via its docker service name caddy_server:8080 and was always getting 0 bytes 200 responses

[segevfiner]

This is very confusing... IMHO it should really return 404 or even not answer at all, just closing the connection. It also happens when you have unrouted paths, not only on unmatching host, and will probably force me to use handle or route with respond so that all unmatching paths fallback to 404 where I could have otherwise also sufficed with just a bunch of reverse_proxy with a matcher.

[francislavoie]

@segevfiner As @mholt said:

Ah, well, that's just how an empty response is written. The semantics are a little nuanced. Basically, it's not that a requested resource wasn't found (404), per-se, it's just that the server wasn't configured to write a non-empty response. 200 means it is working and responded as configured, but empty because it wasn't configured to fill in the response.

You could probably write your config like this to do what you want:

route {
    reverse_proxy /api/* localhost:8080
    respond "Not Found" 404
}

Or...

reverse_proxy /api/* localhost:8080

@notApi not path /api/*
respond @notApi "Not Found" 404

It's up to the user to configure Caddy to handle all requests as they expect. Caddy v2 tries to avoid implicit behaviour (which Caddy v1 had, which made things complicated or hard to override).

[segevfiner]

Yeah exactly. But I have multiple reverse_proxy which I will have to pack together into route or separate handle. Would be nice if there was a simpler way to do such a fallback route with "behavior-first" like config, you had to define a negative matcher to do it like that which is not viable when you have multiple more complicated route matching.

The documentation might want to state this caveat more clearly.

Also the following was very helpful for figuring out the Caddyfile route matching behavior and it might be helpful to incorporate it into the main documentation as well: https://caddy.community/t/composing-in-the-caddyfile/8291

In regards to the argument of no default behavior and being explicit... Well returning and empty response 200 is a default behavior and a confusing one at that. It is probably derived from some Golang default behavior but perhaps a better behavior for Caddy is just closing the connection? Or a configuration for it perhaps?

[segevfiner]

This also loses the ability to use the shorthand syntax of not needing curly braces when you technically have one server and you want to be strict with host matching, since you will have to define another server to catch all requests that don't match the provided hosts to make them return something other than 200.

Basically I end up doing:

http://localhost:8080 {
    ...

    handle {
        respond 404
    }
}

:8080 {
    respond 404
}

Instead of just:

http://localhost:8080

...