Caddy: Rewrite “status” rule unimplemented

[mikepulaski]

1. What version of Caddy are you running (caddy -version)?

0.9.1

2. What are you trying to do?

Use the status rewrite rule in the rewrite directive.

3. What is your entire Caddyfile?

(Put Caddyfile here)

4. How did you run Caddy (give the full command and describe the execution environment)?

caddy

5. What did you expect to see?

Caddy running.

6. What did you see instead (give full error messages and/or log)?

Activating privacy features... done.
2016/09/06 15:44:35 Caddyfile:10 - Parse error: Wrong argument count or unexpected line ending after ‘status'

7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?

echo -e ":2016 {\n rewrite { \n status 200 \n} \n}" | caddy -conf stdin -log stdout

[mikepulaski]

I’ll fork and submit a pull request momentarily.

[abiosoft]

This has been removed here https://github.com/mholt/caddy/pull/1042 to provide a permanent fix for this issue https://github.com/mholt/caddy/issues/1012.

It still works if you're on version <=0.9.1 but starting from 0.9.2 (or if you build from source), you will need to use status directive instead.

[ehyland]

@abiosoft is is possible to use regex path match conditions with the status directive?

I'm trying to achieve the following now it's not possible to use status in rewrites

# block access to php files in mysite, framework and cms
rewrite {
  if {path} match /(mysite|framework|cms)/.*\.(php|php3|php4|php5|phtml|inc)$
  if {path} not_match /framework/.*(main|rpc|tiny_mce_gzip)\.php$
  if_op and
  status 403
}

# block access to other files
rewrite {
  if {path} match /\..
  if {path} match \.ss$
  if {path} match web\.config$
  if {path} match \.ya?ml$
  if {path} match ^/vendor/
  if {path} match /silverstripe-cache/
  if {path} match composer\.(json|lock)$
  if {path} match /(cms|framework)/silverstripe_version$
  if_op or
  status 403
}

[abiosoft]

Try something like this

status 403 /forbidden

rewrite {
  if {path} match /(mysite|framework|cms)/.*\.(php|php3|php4|php5|phtml|inc)$
  if {path} not_match /framework/.*(main|rpc|tiny_mce_gzip)\.php$
  if_op and
  to /forbidden
}

# block access to other files
rewrite {
  if {path} match /\..
  if {path} match \.ss$
  if {path} match web\.config$
  if {path} match \.ya?ml$
  if {path} match ^/vendor/
  if {path} match /silverstripe-cache/
  if {path} match composer\.(json|lock)$
  if {path} match /(cms|framework)/silverstripe_version$
  if_op or
  to /forbidden
}

[ehyland]

That worked, thanks @abiosoft

[kaihendry]

I've updated to 0.9.3 and it has been an ENORMOUS ball ache today that

    rewrite {
            r   /\.(.*)
            status 404
    }

has broken. WHY? :(

[mholt]

@kaihendry Because #1042 was merged, to fix a bug. So just use the new status directive instead. Sorry for the trouble.

[kaihendry]

Breaking the API is always something you really must avoid or people will drop this like a hot potato.

The ultra common use case is to forbid hidden files or folders (stuff prefixed with a dot). https://caddyserver.com/docs/status annoyingly doesn't answer that.

[mholt]

I know breaking changes are inconvenient, I'm sorry about that. I'd rather make them before 1.0 than after, frankly, because they have to be had.

What doesn't answer what exactly?

[kaihendry]

The question is: "How do I stop Caddy serving hidden files?"

You expect to find the answer in https://caddyserver.com/docs/status but I couldn't.

[mholt]

Don't you just specify the files you want to hide, and give them a 404 status?

[kaihendry]

Ok, what is the canonical way to hide '.' prefixed files & directories?

[mholt]

At this point, something like this:

status 404 {
    /.htaccess
    /.hidden
    /.whatever
}

How many dotfiles do you have in your site? O.o

(Don't mind my questions -- trying to gauge the needs of users here.)

[kaihendry]

Isn't it more reasonable just to do all dotfiles?

This is almost the default I think in Apache

<DirectoryMatch "^\.|\/\.">
    Order allow,deny
    Deny from all
</DirectoryMatch>

[mholt]

Maybe for one user, but not for another. I'm just not sure what their needs are yet. (Now I know what yours are at least!)

[kaihendry]

I'm not asking for defaults, I'm asking for documentation that answers the question in a canonical way btw.

[mholt]

Hmm, so like an example? Is what I posted above good for that?

[kaihendry]

You mean https://github.com/mholt/caddy/issues/1092#issuecomment-251284815 ?

No because it's not in the docs and it doesn't handle all dotfiles.

[mholt]

I can't predict all the dotfiles that any user would have :confused:

But thank you for your feedback, I'll look into ways to improve the docs!

[kaihendry]

Match any file with a dot prefix!? === dotfile. What's difficult about that? It was one as easy as r /\.(.*)

[mholt]

I'm confused, status doesn't have an r subdirective. Are you referring to rewrite?

[kaihendry]

Well, it used to work ;) https://github.com/mholt/caddy/issues/1092#issuecomment-250957848

Yes rewrite I guess.

[samwalls]

Hmm, would be great to apply status to regex match: note, this breaks the default Caddyfile for grav. (I'm aware that's not your problem XD)

[technopagan]

@mholt First of all: Thank you for Caddy! It is a beautifully done piece of software & I'm currently moving away from NGINX in favor of Caddy because of its ease of use & sensible default settings.

While I'm aware of the issues of module order execution, I'd still love to see Caddy implement an easy way to cleanly block access to dotfiles while still following other defined rules inside a Caddyfile.

Dotfiles creation inside a site's root is sometimes out of our control, e.g. ".gitignore" in subdirectories or ".etckeeper" can pop into existence without being able to always update a blacklist as suggested here:

status 404 {
    /.htaccess
    /.hidden
    /.whatever
}

Here's how far I've currently come in my attempts to implement this:

# Prevent access to hidden files (.gitignore, .DS_Store etc)
        rewrite {
                if {file} starts_with .
                to error/index.html
        }

# Prevent clients from accessing to backup, config or source files
        rewrite {
            r /(.*).(bak|config|sql|fla|psd|ini|log|sh|inc|swp|dist)$
            to error/index.html
        }

 status 404 error/index.html

While this throws the appropriate status code and shows the defined 404 error HTML page, this approach using rewrite does not honor any custom headers defined for this location block in the Caddyfile.

Expected headers result:

HTTP/1.1 404 Not Found
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Origin: *
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Strict-Transport-Security: max-age=31536000;
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Ua-Compatible: IE=Edge
X-Xss-Protection: 1; mode=block

Actual headers result:

HTTP/1.1 404 Not Found
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Server: Caddy
Vary: Accept-Encoding

Is there a way to implement a clean blocking of access to dotfiles + other defined file types without losing control over response headers?

[mholt]

@technopagan What version of Caddy are you using? And why are those the expected headers, your Caddyfile doesn't have anything in there to set those or remove the Server header?

[Zakay]

I expected also being able disable serving files based on a regex. But only for actual files. If file doesn't exist then it should continue execution as some other rewrite rule might use that path and sends it to another proxy.

edit: Actually even if file exist it should continue execution if there is no handler for that file then just return a 404 as if didn't exist. Or all this might be to complex, easiest would be just return a 404 based on a regex and ignoring what I said earlier :)