Envoy: Symmetric Compression/Decompression between Envoys

[mleventi]

Symmetric Compression/Decompression between Envoys

We have a use case where we'd like Envoy (A) to compress a request body and then Envoy (B) to decompress the request body before sending it upstream. This is useful for situations where there is a cross cloud egress charge boundary between Envoy A and B, request payloads are huge, and the receiving service does not support request body compression natively.

Workflow:
1) Client sends request to Envoy A
2) Envoy A compresses request body and sends request to envoy B.
--- Cloud Boundary
3) Envoy B decompresses request body and sends request onward.
4) Service receives request (uncompressed).

[dio]

I think we can use gzip filter for this. Will find a time to experiment a bit.

[danielhochman]

cc @gsagula

[gsagula]

@dio @mleventi The current GZIP filter does not support decompression. It only compresses data sent from the upstream. If this is only for gzip, it might be reasonable to just enhance the filter.

[mattklein123]

Switching to enhancement. I think we could make the filter also support decompression so that it could be used for symmetrical cases like this. @gsagula any interest in taking this on?

[gsagula]

@mattklein123 Yes, I'm interested in this one! In theory, it shouldn't be too complicated, but let me
take a look at what is needed.

[tonya11en]

I'm also interested in taking this one up. @gsagula let me know if you get busy with other things or need help with review/testing.

[gsagula]

@tonya11en, thank you. Same here! I've wanted to revisit this filter for a long time. Are you interested in just fix this issue or maybe go for a longer stretch? Either way, I would be happy to help with.

[dio]

@gsagula ah thanks for the clarification.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[gsagula]

This issue is being worked based on the discussion here:
https://docs.google.com/document/d/16U0i_X4iO8kqjUtrqMd-LgNEFzwAXzqn4JBH-cOx3Aw/edit?usp=sharing

[derekargueta]

@gsagula any movement here? we're working on an internal decompression filter for gzip

[gsagula]

Hi @derekargueta. I got the compression and decompression working, but it turned out that the filter became really dense. I've started a new filter based on the document above but I didn't have a chance to finish due to work. It's more complex than just gzip compression/decompression though. I'm trying to clear out some ext-authz stuff out of the way before getting back to this one, but it's hard to tell when. I'm happy to help with code review or whatever is not too time-consuming.

[rojkov]

I'm experimenting with decompression filters and here's my branch.

It supports the use case from the issue description and can participate in applying a different compression on an already compressed stream (like "decompress gzip" -> "compress with brotli") when stacked together with another compression filter (e.g. brotli ).

I decided to go with separate generalized filters - one for compression and another for decompression. So far it's been easy to keep the code and configs simple and understandable, but I'm bad at coming up with really complex use cases. Still I can try to merge these two into one universal filter supporting complex policies if it's the preferable way to go.

Though I'm worried that the policies may evolve into stacked filters of a new type (i.e. L4 filters -> http filters -> http compression filters). :)

[rojkov]

Just in case somebody wants to experiment with my branch implementing the described use case here are the sample configs.

Config for HTTP request entry which compresses request body and decompresses response body:

admin:
  access_log_path: /dev/null
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 0
static_resources:
  clusters:
    name: cluster_0
    connect_timeout: 0.25s
    load_assignment:
      cluster_name: cluster_0
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: 127.0.0.1
                    port_value: 10001
  listeners:
    name: listener_0
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 10000
    filter_chains:
      filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: cluster_0
          http_filters:
          - name: envoy.gzip
            config:
              compression_level: best
              compressor:
                compression_direction: request
          - name: envoy.gunzip
            config:
              decompressor:
                decompression_direction: response
          - name: envoy.router
            config: {}

Config for HTTP request exit decompressing request body and compressing response body

admin:
  access_log_path: /dev/null
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 0
static_resources:
  clusters:
    name: cluster_0
    connect_timeout: 0.25s
    load_assignment:
      cluster_name: cluster_0
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: 127.0.0.1
                    port_value: 4500
  listeners:
    name: listener_0
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 10001
    filter_chains:
      filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: cluster_0
          http_filters:
          - name: envoy.gzip
            config:
              compression_level: best
              compressor:
                compression_direction: response
          - name: envoy.gunzip
            config:
              decompressor:
                decompression_direction: request
          - name: envoy.router
            config: {}

And if you need to decompress gzip'ped response and compress it again but with brotli then instead of the first config use this:

admin:
  access_log_path: /dev/null
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 0
static_resources:
  clusters:
    name: cluster_0
    connect_timeout: 0.25s
    load_assignment:
      cluster_name: cluster_0
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: 127.0.0.1
                    port_value: 10001
  listeners:
    name: listener_0
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 10000
    filter_chains:
      filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: cluster_0
          http_filters:
          - name: envoy.gzip
            config:
              compression_level: best
              compressor:
                compression_direction: request
          - name: envoy.brotli
            config:
              compressor:
                compression_direction: response
          - name: envoy.gunzip
            config:
              decompressor:
                decompression_direction: response
          - name: envoy.router
            config: {}

[rebello95]

We're very interested in adapting the gzip filter's functionality to work with Envoy Mobile (ideally via the generalized compression filter(s) described above) as a way of providing an end-to-end compression/decompression pathway for Envoy. I opened up an issue in that repo with some additional info: https://github.com/lyft/envoy-mobile/issues/664

[mattklein123]

This is done.