Runtime: Http response with Content-Length and Transfer-Encoding: chunked headers leads to wrong response string

[AndreyFromGomel]

In general it is not good if there is Content-Length and Transfer-Encoding: chunked headers in single response but on some conditions it may happen (e.g. due to some tools like Charles Proxy or wrong configuration of server endpoint). HttpClient on .Net Core (I checked this on 3.1.200) does not omit Content-Length header if there is "Transfer-Encoding: chunked" in response and "chunked" response is interpreted as "identity". The resulting response string contains additional data (like chunk sizes and line separators).
https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html - p. 4.4.3 says that Content-Length should be skipped if both headers are presented.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding - also mentions that Content-Length has to be omitted in this case.
The same situation now on Xamarin platforms (checked on Xamarin.Android and Xamarin.iOS), but previously they processed correctly this scenario (before CoreFX was merged to mono).
May I expect this behavior will be fixed?

[msftbot[bot]]

Area Owners

@dotnet/ncl

[scalablecory]

If I'm understanding correctly, you're saying this response:

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 12
Transfer-Encoding: chunked

3
foo
0

Will return a string "3\r\nfoo\r\n0\r\n\r\n" from GetStringAsync(), and that you'd instead like it to just return "foo".

[AndreyFromGomel]

Yes.

[scalablecory]

Spec says:

A sender MUST NOT send a Content-Length header field in any message
that contains a Transfer-Encoding header field.

...

If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length.  Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and ought to be handled as an error.  A sender MUST
remove the received Content-Length field prior to forwarding such
a message downstream.

So it seems like we can either treat it as an error or ignore the Content-Length header. Either way, the existing behavior (as described by this issue) is incorrect.

[davidsh]

So it seems like we can either treat it as an error or ignore the Content-Length header. Either way, the existing behavior (as described by this issue) is incorrect.

We should see what .NET Framework and other HTTP stacks do, i.e. WinHTTP or libcurl. We probably should not return an error but rather ignore the 'Content-Length' header.

[AndreyFromGomel]

I tried this on .Net Framework 4.6.1, and Content-Length header is ignored.