Envoy: TCP proxy route by domain name

[poyaz]

Description:
I have two different PostgreSQL instance and i want route by domain name in port 5432 like below:

• if use pg1.example.com (for X application) got to cluster pg1_cluster
• if use pg2.example.com (For Y application) got to cluster pg2_cluster

and i don't know how to use filter or any other options in tcp_proxy

Config:

static_resources:
  listeners:
    - name: postgres_listener
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 5432
#      filter_chains:
#        - filters:
#            - name: envoy.tcp_proxy
#              config:
#                stat_prefix: pg1
#                cluster: pg1_cluster

  clusters:
    - name: pg1_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: pg1_server   # 172.17.0.10
            port_value: 5432

   - name: pg2_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: pg2_server   # 172.17.0.11
            port_value: 5432

[junr03]

This is not a feature of the TCP proxy, nor can it be done with the filter chain match as at L4 there is no knowledge of domain name.

However, if you use the http connection manager you can create L7 aware routing rules like you describe.

[chopmann]

If you use TLS you might get away with using the SNI to do that:

https://www.envoyproxy.io/docs/envoy/latest/configuration/listener_filters/tls_inspector

https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/listener/listener.proto#envoy-api-msg-listener-filterchainmatch

[poyaz]

Sorry, some of the parts for me not understandable.

It's possible to take a sample configuration (how use filter in envoy.yml)?

[chopmann]

Maybe something like this can help

---
static_resources:
  listeners:
    - name: postgres_listener
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 5432
      listener_filters:
        - name: "envoy.listener.tls_inspector"
          config: {}
      filter_chains:
        - filter_chain_match:
            transport_protocol: tls
            server_names:
              - pg1.example.com
          filters:
          - name: envoy.tcp_proxy
            config:
              stat_prefix: pg1_cluster_stats
              cluster: pg1_cluster
          tls_context:
            common_tls_context:
                tls_certificates:
                - certificate_chain: { filename: "...." }
                  private_key: { filename: "..." }
        - filter_chain_match:
            transport_protocol: tls
            server_names:
              - pg2.example.com
          filters:
          - name: envoy.tcp_proxy
            config:
              stat_prefix: pg1_cluster_stats
              cluster: pg2_cluster
          tls_context:
            common_tls_context:
                tls_certificates:
                - certificate_chain: { filename: "...." }
                  private_key: { filename: "..." }

  clusters:
    - name: pg1_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: pg1_server   # 172.17.0.10
            port_value: 5432
    - name: pg2_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: pg2_server   # 172.17.0.11
            port_value: 5432

[poyaz]

sorry, but i can't connect with psql, how to connect with client modules to proxy?

create tls file:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /my-patch/pg1.example.com/tls.key -out /my-patch/pg1.example.com/tls.cert -subj "/CN=pg1.example.com"

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /my-patch/pg2.example.com/tls.key -out /my-patch/pg2.example.com/tls.cert -subj "/CN=pg2.example.com"

psql connection:

psql "host=pg1.example.com user=postgres dbname=postgres sslmode=prefer"

output error:

psql: server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

envoy config file:

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8001

static_resources:
  listeners:
    - name: postgres_listner
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 5432
      listener_filters:
        - name: envoy.listener.tls_inspector
          config: {}
      filter_chains:
        - filter_chain_match:
            transport_protocol: tls
            server_names:
              - pg1.example.com
          filters:
            - name: envoy.tcp_proxy
              config:
                stat_prefix: pg1_cluster_stats
                cluster: pg1_cluster
          tls_context:
            common_tls_context:
              tls_certificates:
                - certificate_chain: { filename: "/my-patch/pg1.example.com/tls.cert" }
                  private_key: { filename: "/my-patch/pg1.example.com/tls.key" }

#        - filter_chain_match:
#            transport_protocol: tls
#            server_names:
#              - pg2.example.com
#          filters:
#            - name: envoy.tcp_proxy
#              config:
#                stat_prefix: pg2_cluster_stats
#                cluster: pg2_cluster
#          tls_context:
#            common_tls_context:
#              tls_certificates:
#                - certificate_chain: { filename: "/my-patch/pg2.example.com/tls.cert" }
#                  private_key: { filename: "/my-patch/pg2.example.com/tls.key" }

  clusters:
    - name: pg1_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: proxy-pg1
            port_value: 5432

    - name: pg2_cluster
      connect_timeout: 1s
      type: strict_dns
      lb_policy: MAGLEV
      hosts:
        - socket_address:
            address: proxy-pg2
            port_value: 5432

[chopmann]

I guess you are be out of luck. Could not find anything useful regarding postgresql and sni.
I dont think this has to do with envoy anymore, and you might want to close this issue.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

[dotw]

@poyaz hello, did you resolve the issue? I have the same one.

[poyaz]

@dotw
no, i can't resolve the issue

[zuercher]

The config pasted above should be sufficient to do SNI based routing of TCP connections made with TLS.

Based on some quick searching, the Postgres protocol starts unencrypted and then upgrades to SSL as part of the protocol. At that point it's too late for SNI to be of use for Envoy's tcp proxy.

Envoy would need an extension that understood at least part of the postgres protocol (and handled upgrade to SSL) in order route connections.

[chopmann]

Community already working on it: #2861

[chintan-mishra]

@dotw I haven't had the time to manage infrastructure in the past few days. Please read the comment below.
https://github.com/envoyproxy/envoy/issues/7066#issuecomment-517898960

The user seems to have solved the issue. I haven't had the time to look and experiment myself