Caddy: v2: UDP/TCP proxying

[mholt]

It would be great if Caddy 2 had a plugin for UDP/TCP proxying, kind of like https://caddyserver.com/v1/docs/net. But v2 makes it possible for this to be even better and more powerful.

If anyone would like to work on this, start by discussing design proposals here. This is something I will be happy to collaborate on with you.

It should be an app module, that is, has Start() and Stop() methods like Caddy's HTTP server. It should also enable TLS automatically and by default, when possible (i.e. when given ServerNames to serve).

Ad-hoc instructions for writing a v2 module are here: https://github.com/caddyserver/caddy/wiki/v2:-Writing-a-Module

/cc @Belak

[colinaaa]

I'm willing to help, and I made a small demo for this:

package caddyproxy

func init() {
    err := caddy.RegisterModule(new(Proxy))
    if err != nil {
        caddy.Log().Fatal(err.Error())
    }
}

type Proxy struct {
    Addr      []Addr
    CacheSize int `json:"cache_size"`
        // other configs

    workers []Worker

    ctx    caddy.Context
    logger *zap.Logger
}

// CaddyModule implement caddy.Module interface
func (Proxy) CaddyModule() caddy.ModuleInfo {
    return caddy.ModuleInfo{
        Name: "proxy",
        New:  func() caddy.Module { return new(Proxy) },
    }
}

func (p *Proxy) Provision(ctx caddy.Context) error {
    p.logger = ctx.Logger(p)
    // init workers
    for _, addr := range p.Addr {
        p.workers = append(p.workers, addr.Worker())
    }
    // other things
    return nil
}

func (p *Proxy) Start() error {
    // start up workers
    return nil
}

func (p *Proxy) Stop() error {
    // close workers
    return nil
}

type Addr struct {
    LocalAddr string `json:"local_addr,omitempty"`
    DestAddr  string `json:"dest_addr,omitempty"`
    Type      string `json:"type,omitempty"`
}

func (a Addr) Worker() Worker {
    switch a.Type {
    case "tcp":
        return tcpWorker{}
    case "udp":
        return udpWorker{}
    default:
        panic("")
    }
}

type Worker interface {
    Forward()
    Receive()
    Close()
}

type tcpWorker struct {
    conn net.TCPConn
    // ...
}

I just get started on caddy 2, and it may take some time for me to get familiar with it.
btw, what do you mean about more powerful?

[mholt]

@colinaaa Great, this is a good start! I don't really understand some of the decisions here but I'm gonna put my questions on hold for a second.

Before going much further, we should probably discuss how we want it to work and what kinds of features we want the app to have.

A few things come to mind:

• The Accept()ing of connections should be "wrappable" (extensible) by modules. This could enable some neat functionality through other modules. For example, a module might accept a connection, read a few bytes, make a decision based on those bytes, then return the connection to the caller. Think like HTTP middleware, but for listeners. (We have something like this in Caddy 1, too. It's how the PROXY protocol plugin is implemented.)

• Connections themselves should be "wrappable" (extensible) by modules as well. This means that modules can wrap what Read() and Write() and other methods do.

• We will need load balancing and health checks, kind of like how the HTTP server does. But of course, these will be implemented at the transport layer, not the application layer.

• It would be really cool if this app could use/embed other apps. For example, perhaps the HTTP app could be embedded by this app, so that a raw TCP connection could be upgraded to HTTP by invoking the HTTP app and passing the connection onto it. Or something.

[belak]

There's some really interesting stuff in https://github.com/google/tcpproxy relating to wrapping connections so you could peek at some of the data. I ran into this when I was playing around with TCP proxying.

[colinaaa]

@mholt Thanks a lot! I totally had no idea what kinds of features this module should have but the "directly proxying". I will try to dig into it now, and I will let you know if I have any questions. Thanks again!

[colinaaa]

@belak Exactly! I noticed the project yesterday, and it's really impressive!

[colinaaa]

@mholt I have made a PR for the first two features you mentioned. But I have no idea how to upgrade a TCP connection to a HTTP connection since I’m not familiar with the caddyhttp module. Could you please give me some instructions about where to go or where to start. Thanks in advance.

[c3c]

Hi, is this still a feature that will end up in caddy? (Either as a plugin or part of the core?)

[mholt]

I am sure it will at some point! Probably as a plugin.

[mholt]

I have implemented a TCP/UDP proxying app for Caddy 2, and will publish it shortly!

[ContainsLiquid]

@mholt That is so awesome to hear. Do you have a timeframe for the implementation?

[francislavoie]

@ContainsLiquid See here: https://github.com/mholt/conncept

[mholt]

Yep, it's working already -- just need to flush out a few more features and stuff. Sponsors get first dibs -- I'm also hoping to reach a sponsors goal and then I'll publish it for everyone to use.

I am already using it on a server. Works very well!

[pwFoo]

@mholt
Would conncept work with lucaslorentz/caddy-docker-proxy?
To have one reverse proxy container to handle all the connections to different containers?

[francislavoie]

Yes it would, since it's just a Caddy plugin. - Docker proxy just builds a Caddyfile from labels and triggers config reloads when necessary.

That said, project conncept doesn't have Caddyfile support yet implemented, but it eventually will if there's enough demand etc.

[mholt]

Oh, I forgot about this issue, but Project Conncept is released now - feel free to give it a spin: https://github.com/mholt/caddy-l4