Caddy: Unable to set header containing JSON in Caddy v2 b15

Created on 4 Mar 2020 · 6Comments · Source: caddyserver/caddy

I'm trying to set the Report-To header to a value that is JSON. I've tried various attempts at escaping braces and quotes but have been unable to set the header value to valid JSON. For more details on the attempts I have tried, see below.

I believe this is a sensible feature request based on increased use of this specification: Reporting API, W3C Working Draft, 25 September 2018.

This feels like a feature request rather than a bug as the documentation doesn't not explicitly indicate that JSON header values are supported. If we don't want to support JSON (or the use of {, }) in header values, we could detail this behaviour in the documentation to clarify this.

I haven't yet tracked down the lines in the source code responsible for parsing header definitions in the Caddyfile. I'll update this issue if I do.

This appears to have been fixed in: #2063 but I believe the behaviour described there is reproducible in the latest beta.

1. My Caddy version (`caddy version`):

v2.0.0-beta.15 h1:Td1esMk7bebftnoBuT3gOqUGxew5HqdIKw3s36S8tNw=

2. How I run Caddy:

a. System environment:

Linux blog 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

I'm using the DIgital Ocean image but have upgraded Caddy to the latest version v2.0.0-beta.15.

b. Command:

I'm using systemctl. The following command is how I reload after a configuration change.

systemctl reload caddy

c. Service/unit/compose file:

cat /etc/systemd/system/caddy.service

[Unit]
Description=Caddy Web Server
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile --resume --environ
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

blog.glvr.io {
  root * /var/www/html
  file_server
  encode gzip zstd

  header {
    -Server
    Referrer-Policy no-referrer
    X-Xss-Protection "1; mode=block; report='https://bill.report-uri.com/r/d/xss/enforce'"
    Report-To "\{'group':'default','max_age':3600,'endpoints':[\{'url':'https://bill.report-uri.com/a/d/g'\}],'include_subdomains':true\}"
    X-Content-Type-Options nosniff
    Content-Security-Policy "default-src 'self'"
  }
}

3. The problem I'm having:

I'm trying to set the Report-To header to a value that is JSON.

{'group':'default','max_age':3600,'endpoints':[{'url':'https://bill.report-uri.com/a/d/g'}],'include_subdomains':true}

I've tried various attempts at escaping braces and quotes but have been unable to set the header value to valid JSON. For more details on the attempts I have tried, see below.

4. Error messages and/or full log output:

~# curl -XGET -I https://blog.glvr.io

HTTP/2 200 
accept-ranges: bytes
content-security-policy: default-src 'self'
content-type: text/html; charset=utf-8
etag: "q6kdmr6l9"
last-modified: Mon, 02 Mar 2020 11:37:39 GMT
referrer-policy: no-referrer
report-to: \],'include_subdomains':true\}
x-content-type-options: nosniff
x-xss-protection: 1; mode=block; report='https://bill.report-uri.com/r/d/xss/enforce'
content-length: 8541
date: Wed, 04 Mar 2020 10:50:52 GMT

5. What I already tried:

I've tried a couple of things:

No escaping at all

config: Report-To "{'group':'default','max_age':3600,'endpoints':[{'url':'https://bill.report-uri.com/a/d/g'}],'include_subdomains':true}"
result: report-to: ],'include_subdomains':true}
Escape all the {

config: Report-To "\{'group':'default','max_age':3600,'endpoints':[{'url':'https://bill.report-uri.com/a/d/g'}],'include_subdomains':true\}"
result: report-to: ],'include_subdomains':true}
Escape the outer {

config: Report-To "\{'group':'default','max_age':3600,'endpoints':[{'url':'https://bill.report-uri.com/a/d/g'}],'include_subdomains':true\}"
result: report-to: \],'include_subdomains':true\}

6. Links to relevant resources:

I've found this previous question that appears to describe the same issue: Header containing JSON.
This question links through to this PR: httpserver.Replacer: Rework loop to ignore escaped placeholder braces which has been merged.
This fix does not appear to have made it into the v2 branch: https://github.com/caddyserver/caddy/blob/v2/modules/caddyhttp/replacer.go

good first issue help wanted

Source

billglover

All 6 comments

I'm not sure if I know how I'd implement this yet, but I'm proposing the following additional test cases:

{
    input:  `{"json": "object"}`,
    expect: "",
},
{
    input:  `\{"json": "object"}`,
    expect: `{"json": "object"}`,
},
{
    input:  `\{"json": "object"\}`,
    expect: `{"json": "object"}`,
},
{
    input:  `\{"json": "object{bar}"\}`,
    expect: `{"json": "object"}`,
},
{
    input:  `\{"json": \{"nested": "object"\}\}`,
    expect: `{"json": {"nested": "object"}}`,
},
{
    input:  `\{"json": \{"nested": "{bar}"\}\}`,
    expect: `{"json": {"nested": ""}}`,
},

billglover on 6 Mar 2020

👍1

Interesting that only the opening curly brace needs to be escaped; I guess that makes sense though. Will probably end up documenting that all need escaping just for consistency/clarity sake, but technically it may not be necessary!

mholt on 6 Mar 2020

This puzzled me a bit. I wanted to implement the check for the closing brace for completeness but it adds complexity and I can't find a test case that justifies it. My logic flipped between these two explanations:

Escaping the opening { is actually escaping the full { ... } structure and not just the { character
I don't need to check for an escaped closing brace because the requirement for a matching brace comes from the fact this is JSON. Caddy doesn't need to know what the string represents and so validation of the closing brace is unnecessary.

I completely agree that for readability it makes sense to escape both in any examples.

billglover on 6 Mar 2020

👍1

Sure, makes sense to me. I can't think of any reason why we'd need to enforce escaping the closing brace, esp. because it's technically more complicated.

mholt on 6 Mar 2020

Hey,

I'm using the json config format, and when I add escaped values to headers, for example:

"report-to": [ "\{test}" ]

I get this error from the validate call

validate: decoding config: invalid character '{' in string escape code

I'm on v2.2.1 h1:Q62GWHMtztnvyRU+KPOpw6fNfeCD3SkwH7SfT1Tgt2c= and https://github.com/caddyserver/caddy/commit/36a6c7daf0f45353efe860e254aa148b7574b04e is tagged as v2.2.1, so shouldn't it contain the fix, or was this fix just for the Caddyfile format?