Envoy: Brotli Compression support in envoy

[nrshrivatsan]

Issue Template

Title: HTTP Compression Brotli support

Description:

Desired behavior

• Envoy is capable for decompressing HTTP payload using Brotli compression/decompression algorithm

Relevant Links:

https://github.com/google/brotli
https://quixdb.github.io/squash-benchmark/#results
https://github.com/google/brotli/tree/master/c
@bmadhavan @unicell @khgandhi @
/new-feature

[htuch]

@jmarantz I've heard you talk about Brotli compression a few times in the past, do you want to weigh in on how this might play out in Envoy?

[jmarantz]

Sure, and I think this is a good idea, with some caveats. Note also that @gsagula created the gzip filter, which should definitely inform the construction of the brotli filter. Arguably, the gzip filter could just be generalized rather than creating a new one, but maybe it's not worth it.

In the past, I've found that Brotli compression offered better compression ratio than gzip at a higher cost of CPU and memory. See past discussion here: https://github.com/apache/incubator-pagespeed-mod/issues/1148 .

From a system perspective, I found in the past that you probably want to use Brotli on cacheable assets, saving the compressed, cached results in the cache, so you amortize the compression costs across multiple requests for the same asset. At least a few years ago, we thought it was better to stay with gzip for non-cacheable assets to avoid CPU/memory consumption issues. That may have changed as there have been a lot of Brotli releases since then: https://github.com/google/brotli/releases

From an Envoy filter perspective it's probably best to ignore that and just create the mechanism to brotli-compress/decompress on-the-fly, and consider imposing system optimization policies as a separate filter.

[nrshrivatsan]

@jmarantz

create the mechanism to brotli-compress/decompress on-the-fly,

Thanks! cc @htuch

[mattklein123]

+1 on generalizing the gzip filter into a general compression/decompression filter.

[valarauca]

+1 on generalizing gzip filter into a general compression/decompression.

Also if zstd could be included, it provides similar trades-offs to brotli as it also uses similar encoding schemes for compressed data.

[nrshrivatsan]

+1 to Generalized (ZSTD is a good contender too!)

One could think about ZSTD too. However for streaming compression benchmarks Brotli seems to be out performing ZSTD. https://quixdb.github.io/squash-benchmark/#results

PS - just parroting the benchmarks. I am not compression algo expert ;-)

[rojkov]

I started to work on this. The current status can be seen in https://github.com/rojkov/envoy/commits/brotli
So far I've managed to hack the gzip filter to use brotli compression instead of gzip.