Fetch: Provide ability to encode content from browser/runtime in fetch API.

[lijunle]

Hi, all.

AFAIK, at this moment, there is no way to ask browser/runtime to encode the request content when calling fetch API.

When handling logging, browser/runtime native gzip encoding is very useful to reduce the bandwidth.

I only need gzip because that is the most common case, but from spec aspect, it is flexible to provide more options.

I am not sure if it is a good options to provide such ability inside fetch API. Maybe another API co-work with fetch API is better.

[annevk]

I guess what would be good to know is at what point in the stack browsers apply compression algorithms. If it's in the networking layer providing instructions in fetch() seems like the way to go. If it's at a higher layer we might want to look at some kind of compression API that can also be reused elsewhere (such as in WebRTC).

cc @martinthomson

[wanderview]

Does the browser implement any compression on uploads currently? My impression is that it was something of a compat issue with servers, so maybe not done.

From a primitives perspective it would be nice to do this like TextEncoder/Decorder operating on ReadableStream. The site would have to use its own mechanism for communicating with the server that its sending a compressed body.

[domenic]

Yeah, in the past @ricea and @tyoshino have talked about doing something similar to https://github.com/whatwg/encoding/pull/127 for compression/decompression algorithms. I think the main thing is to successfully land https://github.com/whatwg/encoding/pull/127 first.

[sleevi]

Correct, Chrome’s networking stack does not have methods or ability to encode uploads. Consumers of the network stack (in the non-Browser cases) are expected to do so. Servers - but also the things between servers (AV, caches, intermediaries) - are bad at this in the general case.

[martinthomson]

There is nothing stopping a user of fetch() from creating a GzipStream as Ben suggests and adding the Content-Encoding header field. And I think that is a fine outcome. Note that this depends on a number of things: knowing that the server supports it (see RFC 7694), and knowing that you aren't mixing attacker-controlled and confidential information in a way that might leak to leaks.

It might not even be prohibitively expensive performance-wise to run gzip or brotli and friends in JS. Having a native implementation is certainly possible. This is something of a niche use case, but there are probably other use cases that might benefit from a general compression mechanism.

[annevk]

It sounds like there's some interest here if someone is willing to work out GzipStream or similar. Such an API could probably live in a subsection of Fetch, but could also be done independently.

[dcramer]

Throwing my two cents in here..

We don't want to have to bundle a version of gzip (or related) inside of an already expensively large bundle of proprietary code in the browser. This is truly something that could be deferred to the browser runtime with hints or a native API. After all, the primary reason to compress something is to save bandwidth.

[domenic]

Oh hey, I think we all forgot about this thread. This has been specced over at https://github.com/WICG/compression.

[annevk]

Okay, so for now the solution will be as @martinthomson outlined above, you set your own Content-Encoding header and encode stuff yourself. I filed https://github.com/WICG/compression/issues/32 to discuss the eventual standards home for Compression Streams.

Once this has been in use for some years we can see if further abstractions are warranted based on popular libraries.