Xamarin-android: Native HttpClient adds redundant headers: User-Agent, Content-Type, Accept-Enconding, Host

[tlaguz]

Steps to Reproduce

var httpClient = new HttpClient { BaseAddress = new Uri("http://10.0.2.2:5100/") };
httpClient.DefaultRequestHeaders.UserAgent.Clear();

var request = new HttpRequestMessage(httpMethod, "test");
request.Content = new ByteArrayContent(stream.ToArray());
request.Content.Headers.ContentType = null;
var result = await HttpClient.SendAsync(request);

1. Make http request using three different HttpClient implementations: Xamarin Native, Xamarin Managed, .Net Core
2. Observe that only Xamarin Native adds User-Agent, Content-Type, Accept-Enconding, Host headers.

Expected Behavior

HttpClient should not add headers: User-Agent, Content-Type, Accept-Enconding, Host when not asked to.

Actual Behavior

HttpClient Xamarin Native adds following headers:

Accept-Encoding: identity
Content-Type: application/x-www-form-urlencoded
User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.012)
Host: 10.0.2.2:5100

Version Information

Android versions tested: 9.0 and 8.1.

Microsoft Visual Studio Community 2017
Version 15.9.7
VisualStudio.15.Release/15.9.7+28307.423
Microsoft .NET Framework
Version 4.7.03190

ASP.NET and Web Tools 2017 15.9.04012.0
ASP.NET and Web Tools 2017

ASP.NET Web Frameworks and Tools 2017 5.2.60913.0
For additional information, visit https://www.asp.net/

Mono Debugging for Visual Studio 4.13.12-pre (9bc9548)
Support for debugging Mono processes with Visual Studio.

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

Xamarin 4.12.3.80 (d15-9@914127c74)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 4.16.13 (45a16efd4)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin Templates 1.1.128 (6f5ebb2)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.

Xamarin.Android SDK 9.1.7.0 (HEAD/ba9da7a76)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 12.2.1.15 (d60abd1)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

[jonathanpeppers]

@grendello I would guess Android is probably doing this, and we don't have control over it. I thought you could take a look, though.

[lesterbotello]

Hi all.

Any updates on this? A similar issue is happening now on our Xamarin.Android project targeting Xamarin.Android 9.4 and above, only difference is HttpClient is addingTransfer-Encoding: chunked as a request header. This is currently causing the remote call to fail.

[grendello]

I don't see the sample code in OP using AndroidClientHandler, just the standard BCL HttpClient (which I assume is referred to as Xamarin Native in the OP). @lesterbotello, is this also what you are using - just the standard .NET HttpClient?

[lesterbotello]

Hi @grendello.

Yes, we're just using the plain standard HttpClient and we're not using any client handlers. It's worth clarifying that this is happening on a method inside a PCL project targeting .NET Standard 2.0, and there's both an iOS and an Android project calling it. Both platform projects exhibit slightly different behavior but in both cases, a request header that is not explicitly added is present in the RequestMessage property of the response when checked on the debugger (for the iOS project is Content-Type, for the Android project is Transfer-Encoding).

[grendello]

@marek-safar It looks like it might be an issue with the BCL HttpClient, mind taking a look?

[grendello]

@lesterbotello thanks! The Mono team will take a look at this issue

[marek-safar]

/cc @steveisok

[steveisok]

@grendello Taking the reproduction steps and running it through the standard BCL HttpClient, I'm only getting (in a POST operation, for example):

POST <address> HTTP/1.1
Content-Length 256 (the size of my byte array)
Host <IP Addr>

I don't find any of those out of bounds. Perhaps there's something else in the pipeline that I'm not aware of?

[lesterbotello]

Hi all.

Just a follow up on this issue. After a lot of debugging, we managed to track our issue to a call to PostAsJsonAsync(). For some reason we couldn't figure out, this was messing up with the request. We got it fixed by serializing the request data using Json.NET and sending it as "application/json" using good-old PostAsync()