Okhttp: Transparent support for Brotli content-encoding.

[AngusMorton]

Brotli content-encoding is supported by the major browsers, and it looks like there is pretty good support on the web server side too.

It could be supported transparently like gzip is, so that when OkHttp receives a request with the header Content-Encoding: br OkHttp uses Brotli decompression transparently.

Brotli content-encoding has reduced our payload size by about 20% for the average response.

[JakeWharton]

How small is a Brotli decoder? We get gzip for free from the Java APIs. I don't see much value in transparent, built-in Brotli unless it can be implemented in a few hundred lines or less so as to not bloat the library. Otherwise a standalone artifact that has a Brotli interceptor seems like a better approach.

[yschimke]

"Brotli uses a pre-defined 120 kilobyte dictionary" - at least that big?

I agree with Jake, even if we want to include it it would be great to be provided to existing users of OkHttp etc as an additional library and turning it on by default is likely to trigger a whole bunch of edge cases for requests against servers that previously just worked.

[AngusMorton]

Yeah, it would be more than a few hundred lines. a separate, optional artifact sounds like the right approach.

[swankjesse]

This is pretty straightforward to do with an interceptor. If anyone is interested in making an interceptor that does Brotli, please make a GitHub project. If/when there's sufficient interest we'll add something to OkHttp itself.

[krombel]

There is currently no java encoder available.
The issue for this is google/brotli#405

[yschimke]

@krombel I don't think that is a hard blocker, just the decoder would be 98% of common usage of this feature. Postel's law would suggest clients accepting brotli compressed content from major services like Google, should come before clients sending brotli compressed to random smaller APIs.

[yschimke]

@swankjesse anything obviously wrong with this impl?

n.b. Consider this particular trivial code as public domain if you want to make a java library on github for it.

https://github.com/yschimke/oksocial/pull/295/files

package com.baulsupp.oksocial.brotli

import okhttp3.Interceptor
import okhttp3.Response
import okhttp3.ResponseBody
import okio.Okio
import org.brotli.dec.BrotliInputStream

/**
 * Transparent Brotli response support.
 *
 * Adds Accept-Encoding: br to existing encodings and checks (and strips) for Content-Encoding: br in responses
 */
object BrotliInterceptor : Interceptor {
  override fun intercept(chain: Interceptor.Chain): Response {
    val request = chain.request().newBuilder().addHeader("Accept-Encoding", "br").build()

    val response = chain.proceed(request)

    if (response.header("Content-Encoding") == "br") {
      val body = response.body()!!
      val decompressedSource = Okio.buffer(Okio.source(BrotliInputStream(body.source().inputStream())))
      return response.newBuilder()
              .removeHeader("Content-Encoding")
              .body(ResponseBody.create(body.contentType(), -1, decompressedSource))
              .build()
    }

    return response
  }
}

[yschimke]

$ ./oksocial https://httpbin.org/brotli
{
  "brotli": true,
  "headers": {
    "Accept-Encoding": "br",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "oksocial/brotli-af876de"
  },
  "method": "GET",
  "origin": "82.5.95.16"
}

n.b. httpbin seems to lie about Accept-Encoding, maybe to make this always return brotli encoded content. println shows I'm sending [gzip, br]

[yschimke]

Closing for now, as no obvious next step for OkHttp as it seems trivial to use an an Interceptor from a library first as Jesse suggests

[swankjesse]

https://twitter.com/addyosmani/status/1130722391120850944?s=19

[swankjesse]

If the claims are correct, we should add okhttp-brotli to package the dependency and interceptor together and keep both up-to-date.
https://github.com/yschimke/okurl/blob/master/src/main/kotlin/com/baulsupp/okurl/brotli/BrotliInterceptor.kt

[yschimke]

It basically "just worked" so that seems good. Zero novel code to use it. But library weight probably means best as an optional dependency.

[yschimke]

A PR, I assume for 4.1

https://github.com/square/okhttp/pull/5088

[swankjesse]

We shipped this in 4.1!