Fabio: Domain or protocol redirection

[mazhack]

Hi again,
i have some problems with fabio, i need redirect http traffic to https, for example
http://domain.com -> https://www.domain.com
http://www.domain.com -> https://www.domain.com

fabio does not support redirection, so i create a dockers with lighttpd and configure it to make the redirections, but i can not add route only for http or https

so if i register docker with tag:
urlprefix-domain.com/ -> one docker, redirects http and https to https://www.domain.com
urlprefix-www.domain.com/ -> register 2 dockers, one for redirect and another for application, redirects http to https, and https redirect again to https, so if i use round robin balancing, half https connections make a redirect again...

it is possible define a protocol for route?, so route http traffic to a group of docker and https to another group of docker, for example:

urlprefix-www.domain.com:http/ -> send to redirect dockers
urlprefix-www.domain.com:https/ -> send to application dockers
urlprefix-www.domain.com:ws/ -> send to websocket dockers
urlprefix-www.domain.com:wss/ -> send to secure websocket dockers

urlprefix-/ -> send http and https, already defined
urlprefix-:http/ -> send only http, new
urlprefix-:https/ -> send only https, new
....

path, domain and protocol are optional, so it is not breaking change.

This is only an idea, a proposal, you have the last word.

[mazhack]

if fabio is behind of another proxy, then this has not sense, but if it is directly connected to internet is very useful

[magiconair]

The way you should do this is to do the redirect in the application and not on the LB. If your application needs a resource to be on HTTPS then it should detect that and send the redirect itself. This way your application does not depend on the LB to do the work.

[mazhack]

I use X-Forwarded-Proto header before, but all domain is secure, so i want to do the redirection out of application, for me is better do it in reverse proxy and not load all php application in symfony for do a redirection.
I am using now 2 fabio instances one for http and another for https and work fine.

many thanks for the answer.

[iadknet]

I agree with @mazhack and am looking for a similar solution.

I understand that service configuration should be decoupled from the load balancer, but something as basic and universal as enforcing ssl definitely belongs with the load balancer.

[magiconair]

The question is what you consider basic and universal but I agree that fabio could provide this providing that the application configures it. That means there needs to be a way for the application to express this. It's already on the list for #87.

[iadknet]

I disagree with the philosophy that this is a service-level configuration.

Services should not have to worry about http/https. That is a product of the environment they live in and should be decoupled from the service itself.

[leprechau]

Termination of SSL/TLS enforcement of that access is extremely important in the LB stack. I would expand this to say that the LB should also allow you to enforce ciphers and acceptable SSL/TLS versions. This way you have a single place to configure your secure service policy.

[magiconair]

@iadknet Services are the only place where you know whether something has to be on HTTP or HTTPS. After all it is your service that defines the requirement and not the environment. What you are doing by moving that configuration into the LB is to split up the service configuration into two places which from that moment on have to be kept in sync. That means you have to configure fabio that this specific route needs to be on HTTPS and that again means that you have to manage that configuration and deploy it everywhere in sync with your service deployments (think puppet). How is that different from any typical nginx/haproxy setup? The whole point of fabio was to not do that but to have the LB auto-configure itself from the requirements the services have. The only way to achieve this IMHO is to have the services tell the LB what to do.

@leprechau One could argue that TLS ciphers are a system wide configuration like the ip address or the port. I would view them as a security configuration and the only situation where I could see them to be configurable is if you have to support some legacy configuration. This feels like and edge case to me but I don't know for sure. In any case that could be made configurable in the static configuration by providing different TLS listeners. With the Go std lib tls stack it isn't possible AFAIR to specify this per connection but only on the listener itself.

[iadknet]

@magiconair I still disagree that ssl is a service-specific configuration. It is an environment configuration. What fabio does to decouple routes from the load balancer config is fantastic. Routes are something specific to the service. But https is a function of the environment, not the service.

Let's say you have a mix of 100 microservices running java, nodejs, and go. When these services come up, all they know is that they start up and listen on a port. If you want each service to be responsible for forcing ssl, then they need the following logic:

does my upstream lb support an ssl endpoint?
   if yes && the request came in on a non-ssl endpoint:
       redirect to the https endpoint

This requires the service to have knowledge of its environment that it shouldn't need to have. How does it know that it is running in an environment that has an ssl endpoint? How does it know what that endpoint is? These are all functions of the environment that should be completely decoupled from the service.

It gets even more complicated because there is inter-service communication that doesn't go through the load balancer (such as consul health checks). So then the logic needs to look like this:

did the http request come through a lb?
   if yes
     if lb request was non-ssl && lb supports ssl
       redirect to ssl
else 
   serve content without redirect

And this logic has to be written into all 100 services, instead of at one entrypoint at the lb.

So, in my environment to solve this problem, I have to have nginx in the mix solely for the purpose of doing https redirects.

(80) AWS ELB -->  nginx (301 -> https)
(443) AWS ELB --> fabio --> microservices

[magiconair]

@iadknet Only your service knows which endpoints need to be on SSL. AWS doesn't and that is the environment IMO. This makes it a service config but lets not split hairs here. I think we've made both our points clear enough :)

Also, I get your other point that it is much simpler to put one rule in front of the app to fix this instead of fixing this everywhere. Don't get me wrong: I'm happy to provide that feature if I can implement it in a way that doesn't require you to provide static config everywhere.

What occurred to me is this: Couldn't you have an https-redirector service which announces http://foo.bar/ and whose only service is to redirect to https://foo.bar/ (with /path) if you want. Then your other services can announce https://foo.bar/. Then you would have to make fabio terminate SSL. Not sure how that works on AWS.

I guess when you say that SSL is an environment config you mean that in some environments you want so use SSL for your app and in others you don't. In that case that's an optimization to make your dev env look different from your prod env.

[iadknet]

@magiconair Yes, I'm sorry to be contrarian :) I agree that ideally whatever the top level entrypoint is into your app (in this case the AWS ELB) should ultimately be responsible for enforcing SSL. Hopefully with the introduction of the ALB, Amazon will add this functionality.

We terminate SSL at the ELB because of the ease of integration with the AWS certificate service.

Fabio, as the router, is the glue that holds all the microservices together and makes them function as a single platform, with a single entrypoint.

To give some insight into how we're using fabio for a multi-tenant platform:
Each service registers with consul and sets their urlprefix tag to register a route.

https://tenant1.domain.com/admin -> (443) ELB -> (9999) fabio -> admin frontend service (serves index.html that references static content in cloudfront/s3)
https://tenant1.domain.com/api/user -> (443) ELB -> (9999) fabio -> user service
https://tenant1.domain.com/api/forms -> (443) ELB -> (9999) fabio -> forms service

For us, SSL is a function of the platform and not of each individual service that composes the platform. Either the platform is SSL terminated or it is not. It might be ignorance on my part, but I can't imagine why you would want to have SSL on some endpoints and not others?

[magiconair]

@iadknet The typical argument was/is that HTTP is inspectable and cacheable and has less overhead than HTTPS. So ideally you guard your auth endpoints and leave the rest open for others to sniff your cookies I guess. :)

Also, no need to worry. I'm happy we're having this discussion since it gives me a better idea whether this is an actual problem or just an idea. The main issue right now is that I don't have a good vehicle to ship the extra options from the urlprefix- tag into the routing table since I've got my intermediary language and a crappy parser. Also, I now have four different representations of the same thing I'm considering replacing the route add ... commands with a compatible JSON structure which I can then use for overrides and on the API. Not as human friendly but probably not the end of the world either and I don't have to write parsers and generators for different representations.

[avarabyeu]

Not sure if it's right place to ask the question, but can i configure redirect from http to https? Seems no, so do you have plans to implement it?

[magiconair]

@avarabyeu It is the right place and right now this isn't possible, unfortunately.

[discobean]

Considering the web is moving to https by default, adding a default redirect to https would make migrating 10s or 100s of microservices behind fabio much easier.

We'd love this feature very much :)

[magiconair]

I'm wondering how we could express this in the syntax. The route command could look like this:

route redirect /foo proto=https code=303

but the urlprefix- tag becomes increasingly useless here.

[kobiakov]

What if redirection would be based on a tag, something like:

route add very-https-service example.com/ http://10.0.0.1:12345/ tags "forcehttps;code=301"

Still unclear though, what to do if multiple https listeners are defined.

[magiconair]

Finally, this got added. Thanks a lot @ctlajoie

[ctlajoie]

You're welcome. Was a pleasure working with you on the PR. 👍

[holtwilkins]

Hey @magiconair or @ctlajoie , can I get an example of the redirect feature that does carte-blanche http -> https redirection?

Namely, what I want is, if you go to http://<host>/<URI>, I want to return you a 301 (or whatever) to https://<host>/<URI>.

It seems like it should be possible now, but I don't want to have to define this redirect for every urlprefix, but ideally can set it in fabio/config so that it applies automatically to all new tags being added. Is this possible?

[leprechau]

It's currently a route setting. You can set it for all paths on a host though.

Example:

route add http-redirect www.example.com:80 https://www.example.com$path opts "redirect=301"

This will redirect any path coming to www.example.com on 80 to www.example.com with an https scheme.

[holtwilkins]

So only all paths on a host, and not for all hosts, eh. I guess this isn't too bad, but a catch-all to set it once and never think about it again would be pretty cool.

[magiconair]

Something like this?

route add https-redirect / https://$host$path opts "redirect=301"

Note that this doesn't work yet but it's easy to add. I'll have a PR ready soon

[magiconair]

@holtwilkins @leprechau https://github.com/fabiolb/fabio/pull/451

This probably needs some more love. I'll have a closer look tonight.

[holtwilkins]

Thanks @magiconair , looks like good discussions happening in the PR, I'd be happy with https-redirect or a :80/ catch-all or whatever lets me achieve my host-and-URI-agnostic http->https redirection :)

[tino]

Am I correct in reading this is only possible as a manual route override, and not in the urlprefix- service tag? Would be nice If I could set this directly on the service definition

[magiconair]

@tino That would be a consequence of this approach. I'm open for suggestions. There is some impedance mismatch between the config language and the urlprefix- tags. Maybe this is a good time to re-open that can again.

Basically, route add commands are generated from the urlprefix- tags. We could add something like fabio <arbitrary command> to the tags which fabio would copy verbatim to the routing table. This way we could keep the short urlprefix- tags but also offer the full power of the config language to the tag based registration.

Then you could add a tag fabio route add :80/ https://$host$path opts "redirect=301" to add this.

Doesn't look like too much work. Any takers?

[magiconair]

Actually, this is trivial ~and would solve probably a lot of problems~. I've written a POC with just a few lines. I'll move this to a separate issue.

Update: I'm not sure if it would actually solve a lot of problems but I would no longer have to maintain two different syntaxes for the same thing. urlprefix- would be a shorthand notation which could also be expressed via route add commands assuming that the values provided by Consul like $service and $addr are available as pseudo-vars. Then urlprefix-/ would be a shorthand for route add $service / http://$addr/ and you could use the latter as Consul tag.