Caddy: [Feature] Service Discovery using Consul

[thibaultmeyer]

Hello,

It could be very nice if Caddy can be dynamically reconfigured by Consul (consul.io).

In a micro-services environment (or not) it allow caddy to get list of available proxy address and path without manual reload.

Nginx have this kind of feature, but only on the "Plus" version. I think it was a killer feature for Caddy.

https://www.nginx.com/blog/service-discovery-with-nginx-plus-and-consul/
https://www.nginx.com/blog/service-discovery-nginx-plus-srv-records-consul-dns/

Thanks

[francislavoie]

In the short-term, check out Traefik. That's what it specializes in. Maybe a plugin would be more appropriate for this feature?

[thibaultmeyer]

Yes i think a plugin could be a good solution. But is plugin can create in-memory caddyfile configuration ? Dont Know if a hook Exist...

[mholt]

I think the rewrite of the proxy will make this possible. (See linked issue.)

[josegonzalez]

Semi-related: https://github.com/brimstone/caddy-consul

[mholt]

@josegonzalez That's cool, what does it do exactly? (ELI5-and-have-never-used-consul)

[josegonzalez]

Seems like a caddy plugin (not that I've used any) that will:

• watch for changes in the consul catalog of services
• build a caddyfile config
• reload caddy

[krishamoud]

Hi everyone, I forked https://github.com/brimstone/caddy-consul for my own use.

https://github.com/krishamoud/caddy-consul is available for anyone who wants to give it a shot. I tried to make it as generic as possible so you can add your own template and the caddyfile will look how you want it to.

[mholt]

Cool. I think this is now possible by reverse proxying to srv:// endpoints. https://github.com/mholt/caddy/pull/1915

[josegonzalez]

Note: For those more familiar with consul, a slightly faster way would be to watch for consul service changes on the local client via stale reads (non-stale reads will go direct to the consul cluster leader). Consul can then invalidate records for services you are watching for _before_ you actually invalidate DNS.

[mholt]

@josegonzalez How trivial is that change to implement, do you think? Is it something that could be a good PR?