Fabio: Sticky Sessions: % traffic and canary

[tecnobrat]

I have read the few issues previously closed regarding stickiness sessions.

Our use case is specifically for canary style deploys. We would like to send a % of traffic to the new version of an application to canary test it with users. We require this to be sticky. Imagine if we send a user to the new deployment with a new feature, and then they refresh the page and the feature is gone!

Without stickiness, the % distribution in fabio isn't really useful. I'd like to hear some feedback on the design decisions that were made to decide on this.

[magiconair]

That depends on your application but I get your point. Our apps are usually backwards compatible the blue/green scenario was to spin up the full set of instances but only send some amount of traffic to them to see whether an issue would show up and then flip.

Sticky routing usually creates more problems but I agree that this could be a useful feature. Need to think about this a bit.

[killcity]

+1

[magiconair]

I think the simplest thing that could work would be to add sticky routing based on the src address or a header, e.g. a sticky=src or sticky=hdr:name option to the route.

[cjdxlgb]

+1

[bodiug]

Canary releasing support in fabio sounds like a really useful feature to us as well.

[magiconair]

Assuming I use a hash function to compute a value for one or more input variables how do I ensure that fabio always routes to the same host without keeping a connection table which has to be shared across all fabio instances? This wouldn't be stateless anymore.

The list of hosts can change at any time and ordering won't help (see example below). To take this approach you'd have to guarantee that a given target host will always get the same identifier and that the hash function will always map to that id if it exists.

Example:

# list of hosts for the given route in some stable order
hosts = ['a', 'k', 'x']

# hash fn for some input params, e.g. src ip
v = fn("1.2.3.4") // e.g. 10

# target host
host = hosts[v % len(hosts)] // 'k'

# now the host list changes 
hosts = ['a', 'b', 'k']
host = hosts[v & % len(hosts)] // 'b' and not 'k'

Another consequence is that during brief times of unavailability of the target host requests would be sent to a different host and then again to the original target host once it comes back. That makes for very unpredictable behavior during times of deployments and upgrades.

fabio would have to try to reach the target host for that session and fail the request if that target host isn't available even though other hosts are available to handle the request since it cannot safely route the request to a different host. Sessions would have to time out...

A good example of why I don't like sticky lb :)

What about a different approach?

The above approach assumes that fabio is doing some magic to route incoming requests to the same target host based on some computed criteria independent of the current cluster configuration. What if we flip this around and make the first instance responsible for providing fabio with enough data to route subsequent requests for the same session to it?

I guess that is what you are already doing when setting a sticky session cookie but the canary use case doesn't necessary involve sticky session cookies. You just want people to be routed to a specific version of the service.

fabio could use a route id header or request param to find the correct target host and it is the responsibility of the app to set that. The instance would have to publish to consul which route ids it handles.

For the canary use case apps could always set a route id with the current version which would ensure that current users stay on version they started on unless there is no instance available with that version. This way you don't need special magic of source ip addresses. You can have clusters of old and new instances and both user groups would remain distinct. You could use this for a/b testing or for real sticky load balancing if each instance has a unique id.

Does that make sense? What do you think?