Caddy: Multiple log outputs

[itskenny0]

1. What version of Caddy are you running (caddy -version)?

Caddy 0.9.0 (+745ae6f Wed Aug 17 16:14:33 UTC 2016)
 1 file changed, 2 insertions(+)
caddy/caddymain/run.go

2. What are you trying to do?

Use multiple log output directives.

3. What is your entire Caddyfile?

domain.com {
        tls [email protected]

        browse
        root /www/domain.com/htdocs/
        gzip
        log / /www/domain.com/access.log "{remote} {when} {method} {uri} {proto} {status} {size} {>User-Agent} {latency}"
        log / /www/domain.com/request.log "{request_body}"
        log / /www/domain.com/headers.log "{request}"
        errors /www/domain.com/error.log
        fastcgi / /var/run/php5-fpm.sock php
}

4. How did you run Caddy (give the full command and describe the execution environment)?

Using supervisord with this config:

[program:caddy]
command=/opt/caddy/caddy
directory=/opt/caddy
autostart=true
autorestart=true
stdout_logfile=/var/log/caddy.log
redirect_stderr=true
user=www-data

5. What did you expect to see?

The different log files being created and logging be performed to them with the respective templates.

6. What did you see instead (give full error messages and/or log)?

The different log files are created, but output is only logged to one of them.

7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?

Use the above Caddyfile.

[tw4452852]

@mholt Maybe we could implement multiple log with one path scope.

[mholt]

@tw4452852 That would be good. Right now this loop assumes just one destination log; we could have it iterate all of them instead.

[tw4452852]

Hi @mholt, I adapt another way(introducing LogEntry), see my commit for details.

[mholt]

Annnnd merged.

[itskenny0]

Thank you for the implementation! :-)

Maybe I'm misunderstanding something, but with my Caddyfile setup like this, the old behaviour (files are created, but nothing written to them) is still present:

        log / /www/domain/access.log "{remote} {when} {method} {uri} {proto} {status} {size} {>User-Agent} {latency}"
        log / /www/domain/requests.log "{when} | {uri} | {request_body}"
        log / /www/domain/headers.log "{when} | {uri} | {request}"

Only access.log is written to.

root@www:/opt/caddy# ./caddy -version
Caddy 0.9.1 (+536daf3 Wed Aug 24 08:30:37 UTC 2016)
 1 file changed, 1 insertion(+)
caddy/caddymain/run.go

Also, a side question. Does Caddy keep an internal ID of some sort for each request? This would be useful to include in the log so one could match different log entries in different log files.

[mholt]

@itskenny0 Strange, the test shows that it does write to multiple logs. I've tried your Caddyfile here locally and it writes successfully to the 3 files...

As for request ID, no, Caddy doesn't do that. I guess I can see why it would be useful; with Go 1.7's new context addition, it's possible without the "hack" of using a header, but I'm going to wait a few months before we make the jump to 1.7-only support.

[tw4452852]

Also I add a test to confirm.

[vanhalt]

Hey there,

I am also trying this and this is the behavior that I see:

$ caddy -version
Caddy 0.9.1 (+e8e5595 Mon Aug 29 16:15:56 UTC 2016)

$ cat /etc/systemd/system/caddy.service

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
Restart=on-failure
User=www-data
Group=www-data
Environment=HOME=/etc/ssl/caddy
ExecStart=/usr/local/bin/caddy -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp
ExecReload=/bin/kill -USR1 $MAINPID
LimitNOFILE=1048576LimitNPROC=64
PrivateTmp=truePrivateDevices=trueProtectHome=trueProtectSystem=fullReadWriteDirectories=/etc/ssl/caddy
CapabilityBoundingSet=CAP_NET_BIND_SERVICEAmbientCapabilities=CAP_NET_BIND_SERVICENoNewPrivileges=true

[Install]
WantedBy=multi-user.target

$ cat /etc/caddy/Caddyfile

something.com www.something.com {
    root /var/www/something.com

    errors {
        log /var/log/caddy-errors.log
    }

    log /var/log/caddy-access.log {
        rotate {
            size 100 # Rotate after 100 MB
            age  14  # Keep log files for 14 days
            keep 10  # Keep at most 10 log files
        }
    }

    log /var/log/caddy-access-verbose.log {
        rotate {
            size 100 # Rotate after 100 MB
            age  14  # Keep log files for 14 days
            keep 10  # Keep at most 10 log files
        }
    }
}

This are the files:

-rw-rw-r-- 1 root www-data 1.8K Sep  3 20:32 /var/log/caddy-access.log
-rw-rw-r-- 1 root www-data    0 Sep  3 20:29 /var/log/caddy-access-verbose.log # I want to put verbose logs here
-rw-rw-r-- 1 root www-data    0 Sep  3 19:34 /var/log/caddy-errors.log

... and when I perform a requests, only the caddy-access.log gets written. What I want is to have some verbose logs with user-agent and stuff like that. Can someone help me out here?

Thanks in advance and sorry for hijacking this issue!

[mholt]

@vanhalt Strange, can't see that here. If you remove log rotation does it fix it? (just shooting blindly here)

[itskenny0]

I have the same issue without log rotation.

@vanhalt, did you omit any lines from your Caddyfile or is that the entire file? I was wondering if we maybe share some directive that causes it.

[vanhalt]

Hey there,

No, those are all the lines that I have in the files. What I want to approach is to have something like "Apache combined logs" and I don't know how to do that with Caddy. So, if you had that even with log rotation... I don't know what to do then.

So, any advice is welcome :)

[tw4452852]

Hi @vanhalt , I think your code base(e8e5595) is old and this feature should work after this commit(536daf3).

[vanhalt]

Sorry for the delay. But how do I get that version? I've tried the getcaddy script and it installed the same version. Downloaded the latest github release and it doesn't show the commit:

./caddy_linux_amd64 --version
Caddy 0.9.1

Sorry for such a newbie question. Thanks in advance!

[mholt]

@vanhalt You'll have to build from source until the next release, which should be soon.

[itskenny0]

@vanhalt Clone the repo and run the build.bash file in the Caddy subfolder (with Go installed of course).