Caddy: Add directive to change plain HTTP port

[hazcod]

Would be nice if we could have a directive to let caddy listen on a >1024 port for the plain HTTP requests.
Currently the vhost:port notation is only for HTTPS traffic, but I want the HTTP listener to listen on >1024 too to circumvent the bind permission denied problem.

Something like http_port 8080 would do fine.

Ideas?

[abiosoft]

Maybe I don't understand your question but Caddy can listen on > 1024 port for both http and https. And no, the vhost:port notation is for both http and https.

[hazcod]

I would like to redirect HTTP to my HTTPS port, but due to a limitation in docker, I cannot let caddy listen on <1024 ports. So I would need two ports; one that I can map to 80 on the host, and one I can map to 443 on the host.

[abiosoft]

If you are not willing to use Let's Encrypt's automatic https, that should be pretty straightforward.

Share your Caddyfile and let us try to see what went wrong.

[hazcod]

Example:

website.com:8080 {
    ...
}

If I now map the host 443 port to 8080 of the container, HTTPS will work. But what about HTTP traffic on port 80? There is no way to map this to another port on the container. Users trying http://website.com/ will get a connection refused.

[abiosoft]

Looks like you're doing it wrong. That is gonna serve HTTP, not HTTPS.

Easy way out is to be explicit.

HTTP -> http://website.com:8080
HTTPS -> https://website.com:8080

The specified scheme will determine what happens.

[hazcod]

Is this what you mean?

website.com:8080 { }
https://website.com:8081 {
    ....
}

Caddy logs;

Activating privacy features... done.
website.com:8080
website.com:8081
website.com:80

And a connection refused on port 80, so not working.

Doing a website.com:8080, https://website.com:8080 gives caddy complaining about there already being a host listening on 8080.

[abiosoft]

Based on that Caddyfile, I do not expect to see website.com:80. Do you have any other server block where the port is not specified ?

Ordinarily, this should be enough.

http://website.com:8080 { ... }
https://website.com:8081 { ... }

[hazcod]

My caddyfile;

http://datapeak.hazcod.com:8080 { }
https://datapeak.hazcod.com:8081 {
    header / {
        "Strict-Transport-Security" "max-age=63072000; includeSubdomains; preload"
        "X-Frame-Options" "DENY"
        "X-Content-Type-Options" "nosniff"
        "X-XSS-Protection" "1; mode=block"
        -Server
    }

   gzip

    proxy / mgmt:8080 {
        #header_downstream -Server
        proxy_header X-Forwarded-For {host}
    }
}

http://dav.datapeak.hazcod.com:8080 { }
https://dav.datapeak.hazcod.com:8081 {
    header / {
        "Strict-Transport-Security" "max-age=63072000; includeSubdomains; preload"
        "X-Frame-Options" "DENY"
        "X-Content-Type-Options" "nosniff"
        "X-XSS-Protection" "1; mode=block"
        -Server
    }

    gzip

    proxy / dav:8080 {
        #header_downstream -Server
        proxy_header X-Forwarded-For {host}
    }
}

@abiosoft , when my caddyfile just contains this;

http://datapeak.hazcod.com:8080 { }
https://datapeak.hazcod.com:8081 { }

I see the same behaviour;

Activating privacy features... done.
datapeak.hazcod.com:8080
datapeak.hazcod.com:8081
datapeak.hazcod.com:80

[mholt]

Just had a moment to take a quick glance at this issue, but I think this is expected. Automatic HTTPS uses port 80 to redirect plaintext traffic to encrypted traffic. If you don't want HTTPS then use tls off or specify http:// in the site address.

[hazcod]

@mholt I do want HTTPS, but I cannot use ports <1024 because else I need to run caddy as root.
In docker, the storage backend AUFS does not support xattr, so no setcap.

I then map the container ports to 80 and 443 on the host.

[abiosoft]

@HazCod With Caddy, there is difference between HTTPS and automatic HTTPS. For now, you need ports 80 and 443 to use automatic HTTPS.

Looks like the option you have now is to specify your certs.

[mholt]

Yes, in that case then you need to use old-fashioned manual TLS.

Now that Caddy fully supports all 3 challenge types (in the 0.9 beta), maybe we can think about a flag to change the HTTP port from 80 to something else.

[thalesfsp]

@mholt make sense :)

[hazcod]

@mholt do you think this will be easy to implement?

[mholt]

@HazCod Yes, probably will. I guess a command line flag registered from the httpserver package should do the trick; then use its value instead of the hard-coded one.

[mholt]

Although, on second thought, we should consider the ramifications of such an option. Changing HTTP from port 80 to something else will break the ACME HTTP challenge.

[hazcod]

Well it will, unless that port is on port 80 externally. Maybe add a disclaimer in the README?

[mholt]

Sure, we can document it on the CLI page. I still don't like making it so easy to shoot yourself in the foot though.

[kilpatty]

Side note: I would really love this feature, as being able to run a docker container as a non-root user means that we can't bind to any protected ports (433 or 80). I would love to be able to run automatic https on ports 8080 and 4433 inside of Caddy, so I can run as a non-root user, even though I'm mapping the ports outside 80->8080 443->4433
:)

[mholt]

The more I think about this the more I don't like it, it seems like a hack to work around a deficiency in some software that frankly you probably don't even need in order to run Caddy, a fully self-contained, static binary.

[hazcod]

The way I see it it's an effective way to not needing to run caddy as root. Personally, I don't feel comfortable doing this until it has at least privilege downgrading.

[ghost]

I run one main caddy on the host and proxing inside my containers running also caddys... where's the problem using this approach?

[stp-ip]

In my opinion it would be nice to let users tell their default http and https port via cli. It might be, that all sites need to run on 8080 and one would dislike having to type out every :8080. Or the aforementioned non-root issues.

Why not just have something like: ./caddy -default-http:8080 -default-https:8443. This would let caddy listen on these ports by default even for the https challenge.

example.com {
}

^ this would listen on :8080 and :8443 easily without setcap and root privileges and also enable https through these ports.

[mholt]

While this is possible, are the implications of this clear? You _will_ need ports 80 and 443 to solve the ACME challenges, unless you configure the DNS challenge. Right now that's basically a given. But then you change the default HTTP and HTTPS ports because you can't bind to 80 or 443 and then suddenly your sites break when your certs expire because they require ports 80 and 443 (unless you configure DNS too).

[stp-ip]

I think the implications are quite clear. One reason to expose this as cli instead of a directive in my opinion. This is a workaround in my opinion for setups with standard ports not usable. Be that due to running as non-root in containers or in enterprise setups, where multiple ports are already bound.

Not saying it makes sense to jump into this immediately, but should be considered in my opinion. Right now in my specific use cases either the startup process runs as root or caddy itself is, which is not perfect. Still trying to get a workaround working for kubernetes without using root to set the bind capability.

Thanks for taking the time to let us reason around this even so it might not be considered standard use case.

[stp-ip]

Considering the effort I went through tracking the possibilities to deescalate the permissions in different environments (I have to admit mainly chroot and containers) the sane way would be to use setcap, which in itself needs root access. Making it possible to bind to higher ports and doing the translation to the lower ports on the scheduler/NAT etc. side might make sense. This would enable starting up caddy without any need for root.
So there are various ways to address this:

• setcap + root for starting up
• root (not every environment supports setcap)
• suggest DNS challenge as a workaround
• make ports configurable

As said above, I might now consider switchable ports the easiest solution. For any feedback how to deescalate better, I am all ears.

[hazcod]

@stp-ip : mind you that setcap will not work on all docker storage backends.
(Aufs (the default) does not support xattr)

[stp-ip]

Yeah that's what I meant with not supported everywhere. Currently looking into using capsh and deescalation on that route, but that's not the discussion here.

[stp-ip]

For informations sake this is were I am stopping for today:

Still needs root to run capsh, but circumvents the xattr support issue or at least I think it should.
capsh --keep=1 --user='www-data' --caps='cap_net_bind_service=+ep' --inh='cap_net_bind_service' -- -c '/usr/bin/caddy --quic --conf=/etc/config/caddyfile'
This will spawn caddy as a unprivileged user with the added cap (not yet working on my side so).
Then you also need to take care of the /root/.caddy path and also make /root/.ssh writable to the "www-data" user, when using git.

Perhaps that helps someone until we figure out, what the best solution is.

[stp-ip]

This seems to work with a from source build of libcap:

capsh --caps='cap_net_bind_service+eip cap_setpcap,cap_setuid,cap_setgid+ep' --keep=1 --user='www-data' --addamb='cap_net_bind_service' -- -c '/usr/bin/caddy --conf=/Caddyfile'"

Addition:

As I am using Kubernetes the route I am now using is basically to run caddy as root with every capability dropped besides CAP_NET_BIND_SERVICE via the container runtime.

    spec:
      containers:
      - name: caddy
        image: seetheprogress/caddy-extra:dev-e90dc9
        securityContext:
          runAsUser: 0
          readOnlyRootFilesystem: true
          capabilities:
            drop: ["all"]
            add: ["NET_BIND_SERVICE"]

So immutable root filesystem with mounts at /tmp, /root and /config to enable git and temp files to work.
Horrible kubernetes file, simple container, at least a hugely reduced attack surface.

[hazcod]

Hi @mholt, what is the current stopper for the PR?

[mholt]

@HazCod This is an issue, not a PR -- but if you're asking about the previous PR linked to this issue: it was implementing a change that was actually orthogonal to this issue. This issue isn't about making ACME ports configurable, it's about changing the port that Caddy considers the plaintext HTTP port (currently hardcoded as 80). So we simply need a new PR that sets a variable to 80 and has Caddy use that variable in all relevant places. :)

[hazcod]

https://github.com/HazCod/caddy/tree/patch-1 but i'm still trying to compile my first go project. This might take a while ;-)

[mholt]

@HazCod Nice -- not too far off! I've left a comment inline for you to review.

[FiloSottile]

Chiming in: I'm on OpenBSD where I don't have cap_net_bind_service, so I have to redirect ports with pf instead.

However, http_port sounds confusing to me. The result would be:

https://example.com:4443 {
    http_port 4080
    listen 127.0.0.1
}

And the result will be that Caddy binds on 127.0.0.1:4080 and 127.0.0.1:4443, thinking the host is example.com:4443 when it's actually example.com.

I like @tdsmith's patch. The HTTPS port should definitely be configurable, too.

Secondarily, if I use the DNS challenge and don't want the redirect, the HTTP port should be possible to disable.

[FiloSottile]

Looking a bit more at @tdsmith's patch, I think the command line option should not just change the ACME port, but also the default listening and redirect one.

I'll give a patch a try.

[hazcod]

@mholt I tried writing my first bits, only thing left is getting this config from httpserver.go to the tlsConfig variable in the server.go/NewServer function. Ideas?

https://github.com/mholt/caddy/compare/master...HazCod:patch-add-acmeport

[mholt]

@HazCod The name acmePort is misleading; ACME uses multiple ports for different reasons. Try calling it HTTPPort instead (and the flag being httpport). It really has nothing to do with ACME, it's just the port that Caddy considers to be the "default" port for HTTP.

[hazcod]

Yeah sorry, I currently don't have enough time to learn myself go for this PR. Just wanted to let you know.. :(

[mholt]

Thanks @HazCod, I appreciate the effort anyway!

[hazcod]

Thanks again @mholt, can finally run it as non-root! :D