Runtime: HttpClient no longer sends header 'Accept-Encoding:"gzip, deflate"' and decompress 'gzip/deflate' content by default in .NET Core 2.0

[daxian-dbw]

This is a breaking change compared to netcoreapp1.1.
In netcoreapp1.1, HttpClient automatically sends header "Accept-Encoding": "gzip, deflate" in the request, and automatically decompress 'gzip/deflate' content from the response.
However, with netcoreapp2.0, HttpClient doesn't send the header and stop automatically decompressing 'gzip/deflate' content.

Repro Steps

```c#
// Program.cs
using System;
using System.Net.Http;

namespace test
{
class Program
{
static void Main(string[] args)
{
using (var client = new HttpClient())
{
using (var request = new HttpRequestMessage(HttpMethod.Get, @"http://httpbin.org/headers"))
{
var result = client.SendAsync(request).Result;
var payload = result.Content.ReadAsStringAsync().Result;
Console.WriteLine("Header Dict:\n{0}", payload);
}

            using (var request = new HttpRequestMessage(HttpMethod.Get, @"http://httpbin.org/gzip"))
            {
                var result = client.SendAsync(request).Result;
                var payload = result.Content.ReadAsStringAsync().Result;
                Console.WriteLine("Decompressed Content:\n{0}", payload);
            }
        }
    }
}

}

**With netcoreapp1.1**
`http://httpbin.org/headers` just echos the request headers. You can see `Accept-Encoding` was sent by `HttpClient`.
`http://httpbin.org/gzip` returns gzip-encoded data. You can see the content was automatically decompressed by `HttpClient`.

PS> dotnet restore
PS> dotnet run
Header Dict:
{
"headers": {
"Accept-Encoding": "gzip, deflate",
"Connection": "close",
"Host": "httpbin.org"
}
}

Decompressed Content:
{
"gzipped": true,
"headers": {
"Accept-Encoding": "gzip, deflate",
"Connection": "close",
"Host": "httpbin.org"
},
"method": "GET",
"origin": "131.107.147.177"
}

**With netcoreapp2.0**
`Accept-Encoding` was not sent by `HttpClient` and the gzip-encoded data was not automatically decompressed.

PS> dotnet restore
PS> dotnet run
Header Dict:
{
"headers": {
"Connection": "close",
"Host": "httpbin.org"
}
}

Decompressed Content:
 W??X?%?A
?0E?9E???
nE? ^??C2????K??]??????r?.V?O?????[5?rJ8qR?+?>???U?"?E?q ??v?>(?????
.h7??;??????
??

Environment
---------------
**netcoreapp1.1**

.NET Command Line Tools (1.0.1)

Product Information:
Version: 1.0.1
Commit SHA-1 hash: 005db40cd1

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: ...\AppData\Local\Microsoft\dotnet\sdk\1.0.1

**netcoreapp2.0**

.NET Command Line Tools (2.0.0-preview1-005724)

Product Information:
Version: 2.0.0-preview1-005724
Commit SHA-1 hash: e391b5d5f3

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: ...\AppData\Local\Microsoft\dotnet\sdk\2.0.0-preview1-005724\
```

[davidsh]

Yes, it is a change from .NET Core 1.1.

But it aligns with .NET Framework behavior. And one of the goals of NETStandard2.0/.NET Core 2.0 release is app-compat alignment with .NET Framework.

cc: @stephentoub @karelz @CIPop

[davidsh]

See PR dotnet/corefx#17701

[daxian-dbw]

@davidsh Thanks for the clarification. This change breaks our tests, but I think it makes sense.

[davidsh]

Closing as by-design.

[karelz]

In theory, we could view it as improved behavior in .NET Core over .NET Framework, right? We could eventually port the improvement into .NET Framework (netfx-port-consider).
We won't stop innovating in .NET Core, just because .NET Framework cannot catch up immediately.

The question should be: Do we envision this API enhancement in the platforms in a few years? Is it valuable enhancement?
If it is very low value, I am fine either way.
If it is convenience and innovation of our stack(s), I would vote just to keep it in .NET Core and prioritize compat with 1.1 with lack of features in .NET Framework. .NET Framework can eventually catch up.

@davidsh what is your take on that? (reopening to finish the discussion for 2.0)

[karelz]

cc @geoffkizer

[davidsh]

@davidsh what is your take on that? (reopening to finish the discussion for 2.0)

It is highly unlikely we would ever port it to .NET Framework because it would break app-compat. The bar is very high there and no breaking changes are allowed. This would be a breaking change of a default value of a property. I've never seen that allowed by NETFX shiproom or app-compat review.

There is no missing feature here. It is simply the default setting. It can be mitigated by changing a property.

[davidsh]

Also, not sure why this is marked as "tenet-compatibility". It is already compatible with .NET Framework at this point. That label is only used when it is not compatible with .NET Framework.

[karelz]

That label is only used when it is not compatible with .NET Framework.

That is not correct, it is used for any compatibility with any product we consider important. Compat with 1.x is also important. And it is not black and white :(

Based on your information - it is a change of default value, not something we want to port to .NET Framework ever, I personally agree with your decision to keep it compatible with .NET Framework. ... Closing, thanks for explanation @davidsh!
Also I assume there is an easy workaround to compensate for different default value, right?

@geoffkizer @stephentoub please speak up if you disagree. Thanks!

[karelz]

@davidsh can you please document the incompatibility here? https://github.com/terrajobst/platform-compat/issues/21

In a form: API X behaves this way on 1.x and that way on 2.0/.NET Framework
Thanks!

[davidsh]

@karelz

Are you sure that is the correct issue to log compat issues? The issue discusses legacy/deprecated things:

"Steer developers away from legacy/deprecated APIs dotnet/corefx#21"

Perhaps there is another issue I should document this into?

[davidsh]

Summary:

On .NET Core 1.1, the default value for the HttpClientHandler.AutomaticDecompression property was

```c#
DecompressionMethods.Deflate | DecompressionMethods.GZip

On .NET Core 2.0, the default is now:

```c#
DecompressionMethods.None;

This was changed so that it is now the same as .NET Framework.

If you need to change the default back to what it was in .NET Core 1.1, use the following code:

c# var handler = new HttpClientHandler(); handler.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip; var client = new HttpClient(handler);

[davidsh]

Perhaps I need to create a new issue in that repo? If so, I created this new issue for tracking:

https://github.com/terrajobst/platform-compat/issues/22

[karelz]

Yeah, that original issue was not best. The new one is overkill, but we will eventually migrate to some database (at least on wiki).
Thanks @davidsh!

[NightGreyMoon]

Summary:

On .NET Core 1.1, the default value for the HttpClientHandler.AutomaticDecompression property was

DecompressionMethods.Deflate | DecompressionMethods.GZip

On .NET Core 2.0, the default is now:

DecompressionMethods.None;

This was changed so that it is now the same as .NET Framework.

If you need to change the default back to what it was in .NET Core 1.1, use the following code:

var handler = new HttpClientHandler();
handler.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
var client = new HttpClient(handler);

Same issue mess me up in .NET Core 3.1, thanks davidsh's summary, it works for me!