Envoy: Testing istio route rules with envoy side car and didn't work for the first 10+ refreshes

[linsun]

my Istio route rule started with all traffic goes to bookinfo review-v1. Then I applied 50% traffic goes to bookinfo review-v3, e.g. istioctl replace -f route-rule-reviews-50-v3.yaml.

The new rules are like below:

→ kubectl describe routerule reviews-default Name: reviews-default Namespace: default Labels: <none> Annotations: <none> API Version: config.istio.io/v1alpha2 Kind: RouteRule Metadata: Cluster Name: Creation Timestamp: 2017-09-21T20:59:38Z Generation: 0 Resource Version: 49124 Self Link: /apis/config.istio.io/v1alpha2/namespaces/default/routerules/reviews-default UID: c7cb4732-9f0f-11e7-b3d0-22d9bf5954cc Spec: Destination: Name: reviews Precedence: 1 Route: Labels: Version: v1 Weight: 50 Labels: Version: v3 Weight: 50 Events: <none>

What I noticed is that I hit at least 10 times on the browser, however, all traffic goes to the reviews v1. I posted this in our #pilot channel and started some discussion about it. I was told that this is a feature from envoy where it is 50% for the 100 requests so it is possible the first 10+ all goes to v1. Then I started to hit browser refresh again and started to see some traffic goes to v3.

The link below contains my curl output.

https://gist.github.com/anonymous/e1fe93dbe87bfb5dc9d6525568686708

When you see red is returned, it means it goes to review v3. no red means review v1. You can see after 18 retries, I start to see reviews v3 (red). This is very misleading as I would think it is not working until I'm patient to hit the 19th and 20th refresh. Is there a way to make it more look like 50%?

[ldemailly]

@rshriram mentioned the implementation (pointers welcome to confirm) is basically based on comparing streamID with the requested weights

if instead a lookup tables was made according to the weights (for instance for 15% v1, 30% v2, 55% v3 you fill an array of 100 pointers with 15 v1., 30 v2, 55 v3 and then shuffle the array (once) then use the array with Id%100 you get O(1) and the expected user experience both at very low and very high QPS

[mattklein123]

I'm not exactly sure what we are talking about here. The weighted selection is based on randomness: https://github.com/envoyproxy/envoy/blob/master/source/common/router/config_impl.cc#L315

There is a gotcha here. For performance reasons, stream_id is sometimes random and sometimes sequential and is currently based on usesRuntime(): https://github.com/envoyproxy/envoy/blob/master/source/common/router/config_impl.cc#L509
https://github.com/envoyproxy/envoy/blob/master/source/common/http/conn_manager_utility.cc#L21

For now, add a runtime route rule in Pilot as a hack in the route table and it will move stream IDs to a random #. We will need to figure out some better solution here. Maybe if weighted rule is in effect, usesRuntime() should be true.

I will assign this over to @rshriram to look further.

[mattklein123]

Closing out as fixed.

[ldemailly]

fixed ? where/how ?

[mattklein123]

https://github.com/envoyproxy/envoy/pull/1878

[ldemailly]

nice, thanks