Contour: Header based routing

[pbarker]

Expose header based routing https://www.envoyproxy.io/docs/envoy/v1.5.0/api-v1/route_config/route#headers

[davecheney]

@pbarker thanks for raising this issue. Could you describe a little more about what you need this for. Said another way, if Contour let you route on a header, rather than the URL path, what would that enable you to do?

[pbarker]

So, our specific use case is wanting to route to 'feature pods' based on a header. This would entail deploying a downstream service with a new feature and wanting to run tests on it before generally routing traffic to it. Not sure if thats in the scope for contour

[stevesloka]

@pbarker so you would want something like this? Say you have 2 services in your cluster named service-foo and service-bar.

Requests to example.com with no header would route to service-foo and requests to example.com with a header of X-Contour-App: bar would route to service-bar?

I think this could be doable by adding a header to the DAG. Right now Contour builds a dag based upon the fqdn + path (e.g. example.com + /). We could take into account a header as well which would make the three unique. Any delegate of the root IngressRoute that has a header match would also need to match a header (if applicable).

[pbarker]

awesome @stevesloka, love a good DAG. I'm also wondering on a general level whether contour would consider supporting a plugin ecosystem. Has that idea been put out there?

[davecheney]

I'm also wondering on a general level whether contour would consider supporting a plugin ecosystem. Has that idea been put out there?

I'd like to avoid plugins as long as possible.

[rosskukulinski]

But that's not to say plugins are off the table, especially if there are additional engineering and/or PM resources as well as sufficient user-story & design docs? Thoughts @davecheney?

[davecheney]

I'm sorry to be so firm, but Go plugins are weakly supported (linux yes, macos, sorta, windows, nope, lets not even talks about non intel platforms) and have incredibly tight coupling because the plugin and the program its being plugged into are built from an identical toolchain. The developer who originally added plugin support to Go (for the Google Fuschia project) left Google and not one jot of engineering effort has been spent on plugins since. I can't use plugins as a basis for Contour, it's too risky.

[davecheney]

With that said, I'm not saying no to header based routing. What has to happen is we have to write up the proposal as a design document, once we have a design we'll have a guide how much work it will take and can figure out how to fit it in the schedule.

As with every feature we add to Contour, the sticking point is always exposing it Ingressroute in a way that is simple to use and hard to misuse.

[davecheney]

Here's a strawman design

spec:
  routes:
  - match:
      header: 
        name: "x-header"
        value: "A"
      services:
      - name: backend-a
         port: 9999
  - match:
      header:
         name: "x-header"
         value: "B"

The match key deprecates the prefix key. To replace it we'll add

spec:
  routes:
  - match:
       prefix: /quux
     services:
       name: ftang
       port: 9999

[pbarker]

Thanks for the early design!

I'm sorry to be so firm, but Go plugins are weakly supported

Sorry mixed up my terminology, by plugins I'm referring to envoy c++/lua filters https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/http_filters#config-http-filters

[davecheney]

The concern I have with supporting Lua or other arbitrary plugins is support for helping people write Lua filters then falls inside the boundary of what Contour supports towards our community and our essential PKS users. Sure, we could explicitly call out that we can't help with _using_ the feature we added, but people aren't going to realise that until they hit a problem, ask for support, then are told we don't support that. Given they might be paying us for essential PKS support, that's going to be a difficult conversation, and one I'd like to avoid by only adding features that we can support fully.

Thinking laterally one could deploy Envoy as a stright k8s service with a static configuration file (it does support static configuration -- we call it bootstrap, but anything you can serve dynamically you can encode into a static configuration file; it's all serialised protobuf at the end of the day) with your configuration and your lua filtering. You could even connect this dedicated Envoy to Contour for _some_ of the xDS tables Contour can serve, i'm thinking EDS which will give you the dynamic endpoints of the k8s service you're trying to route too.

So, this is possible to do, but a terrible way to spend your innovation tokens. If you can describe can describe your header routing requirements, maybe we can add that as a first class feature to Contour, and CRE will stand behind it to boot.

[davecheney]

Explicit support for Lua filtering is #1015

[stefanprodan]

Hi @davecheney would you consider a similar routing definition to Istio, Ambassador and Gloo?

This is how it could look:

spec:
  routes:
    match:
    - headers:
        user-agent:
          regex: ^(?!.*Chrome).*Safari.*
      uri:
        prefix: /app
    - headers:
        cookie:
          regex: ^(.*?;)?(user=test)(;.*)?$
      uri:
        prefix: /app
    rewrite:
      uri: /

[davecheney]

Maybe, but I’m wary of encoding too much application specific based logic into envoy as a load balancer. It’s role is always a proxy, to direct traffic, not to process it directly.

I’m pretty gunshy about using regex because regex flavours differ between different specs and implementations. In general that’s a non issue, until it’s absolutely an issue and there is zero wiggle room to change what k8s or envoy define, independently, as a regex.

On 12 Apr 2019, at 19:10, Stefan Prodan [e-mail] wrote:

Hi @davecheney would you consider a similar routing definition to Istio, Ambassador and Gloo?

This is how it could look:

spec:
routes:
match:
- headers:
user-agent:
regex: ^(?!.Chrome).Safari.*
uri:
prefix: /app
- headers:
cookie:
regex: ^(.?;)?(user=test)(;.)?$
uri:
prefix: /app
rewrite:
uri: /
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[stefanprodan]

I think regex is a "must have" feature to properly support A/B testing. I was looking at making Flagger work with Contour for this use case https://docs.flagger.app/usage/ab-testing

[davecheney]

Thank you. I appreciate that feedback. The simple answer is regex is powerful, so I want to be very careful where and how it is used. I'm not saying no, I'm just saying I want to cautious because the regex genie doesn't go back inside the bottle.

[pbarker]

Also just hit an issue with regex here, where we need to map to nested services: /foo/<name>/bar where I need bar to go to a different service e.g. /foo/(.+)/bar.

[davecheney]

To be clear. I'm not saying we will not add header based routing to Contour. What we will not do is add header based routing without first establishing the use cases that it solves.

[aaroniscode]

User Story: VMware is working with a global financial company that has incoming traffic that first passes through an authenticating proxy that injects the info about the authenticated user into the header. Traffic is then passed to a request router, that we are looking to replace with a Kubernetes Ingress. That router inspects the header and uses the information to direct to a specific customer service in a namespace.