Envoy: in http2 side, why envoy make more than 1 connection between envoy-cluster?

Created on 17 Jan 2018 · 4Comments · Source: envoyproxy/envoy

Issue Template

Title:

in http2 side, why envoy make more than 1 connection between envoy-cluster?

Description:

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/connection_pooling.html?highlight=connection%20limit

as I know is, in http2 side, there is only 1 connection between envoy-cluster, but I saw in wireshark that envoy make more than 1 connection between them.

like below

I sent to envoy listener[port: 8000] with 2 queries. I expect envoy make only one connection between
envoy and cluster[port:20001] (I enforce envoy config to 1 stream per 1 connection )

as you can see, envoy make 2 connection between [port: 20001] cluster.

below is my envoy config

Config:

admin:
access_log_path: /home/wj/test/envoy/log.log
address:
socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:

listeners:

name: listener_0
address:
socket_address: { address: 127.0.0.1, port_value: 8000 }
filter_chains:
- filters:
  - name: envoy.http_connection_manager
    
    config:
    
    codec_type: HTTP2
    
    idle_timeout: 1s
    
    stat_prefix: downstream_cx_http2_total
    
    route_config:
    
    name: local_route
    
    virtual_hosts:
    - name: local_service
      
      domains: ["*"]
      
      routes:
      - match:
        
        prefix: "/"
        
        route: { cluster: en-US_Service }
        
        http_filters:
    - name: envoy.router

clusters:

name: en-US_Service
max_requests_per_connection: 1
connect_timeout: 10s
cleanup_interval: 10s
type: STATIC
lb_policy: ROUND_ROBIN
http2_protocol_options:
hpack_table_size: 4096
max_concurrent_streams: 1
circuit_breakers:
thresholds:
- max_connections: 100
  
  max_pending_requests: 100
  
  max_requests: 600
  
  hosts: [{ socket_address: { address: 127.0.0.1, port_value: 20001 }}]

name: ko-KR_Service
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
http2_protocol_options:
hpack_table_size: 4096
max_concurrent_streams: 2147483647
circuit_breakers:
thresholds:
- max_connections: 4096
  
  max_pending_requests: 6000
  
  max_requests: 6000
  
  hosts: [{ socket_address: { address: 127.0.0.1, port_value: 8084 }}]

question

Source

hehe717

Most helpful comment

Not sure if this one helps:

All connection pools in Envoy are per worker thread. So, though HTTP/2 connection pools only make a single connection to each upstream host at a time, if there are four workers, there will be four HTTP/2 connections per upstream host at steady state.

From https://blog.envoyproxy.io/envoy-threading-model-a8d44b922310

dio on 17 Jan 2018

👍2 ❤1

All 4 comments

Not sure if this one helps:

All connection pools in Envoy are per worker thread. So, though HTTP/2 connection pools only make a single connection to each upstream host at a time, if there are four workers, there will be four HTTP/2 connections per upstream host at steady state.

From https://blog.envoyproxy.io/envoy-threading-model-a8d44b922310

dio on 17 Jan 2018

👍2 ❤1

@dio is correct, the connections are per-worker. If you don't need the concurrency you can run with less workers which will make less connections.