Xamarin-android: [Android] ServerCertificateValidationCallback not called anymore

[DevEddy]

Since the last Visual Studio for Mac update ServerCertificateValidationCallback is not called anymore on Android (iOS is working as expected). Have I missed something or is it actually a bug?

Steps to Reproduce

1. Configure ServicePointManager

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback = ValidateServerCertficate; 
static bool ValidateServerCertficate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
// ....
}

1. Make a HTTPS-Request with HttpClient

var httpClient = new HttpClient(new HttpClientHandler()) { Timeout = TimeSpan.FromSeconds(10) };
await httpClient.PostAsync("https://.....", content)

Expected Behavior

ValidateServerCertficate is called to validate the certificate.

Actual Behavior

  at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore (System.IO.Stream stream, System.Net.Security.SslClientAuthenticationOptions sslOptions, System.Threading.CancellationToken cancellationToken) [0x00102] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at System.Threading.Tasks.ValueTask`1[TResult].get_Result () [0x0001b] in <ce98bdbc79264bc29c49faa4bba4b921>:0 
  at System.Net.Http.HttpConnectionPool.CreateConnectionAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00322] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at System.Threading.Tasks.ValueTask`1[TResult].get_Result () [0x0001b] in <ce98bdbc79264bc29c49faa4bba4b921>:0 
  at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync (System.Threading.Tasks.ValueTask`1[TResult] creationTask) [0x000a2] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at System.Threading.Tasks.ValueTask`1[TResult].get_Result () [0x0001b] in <ce98bdbc79264bc29c49faa4bba4b921>:0 
  at System.Net.Http.HttpConnectionPool.SendWithRetryAsync (System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) [0x00089] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at System.Net.Http.RedirectHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x000ba] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at System.Net.Http.HttpClient.FinishSendAsyncBuffered (System.Threading.Tasks.Task`1[TResult] sendTask, System.Net.Http.HttpRequestMessage request, System.Threading.CancellationTokenSource cts, System.Boolean disposeCts) [0x0017e] in <4c5165bb22f249eb99ce471318f94f2b>:0 
  at CloudMatic.Connection.Services.ServicePointConfiguration.GetValidHashesFromKeyPinningServer () [0x0008f] in /Users/eschaefer/Documents/GitHub/CloudMatic/Shared/CloudMatic.Connection/Services/ServicePointConfiguration.cs:87 

  The SSL connection could not be established, see inner exception.

  Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at /Users/builder/jenkins/workspace/archive-mono/2019-06/android/release/external/boringssl/ssl/handshake_client.c:1132

Version Information

=== Visual Studio Community 2019 for Mac ===

Version 8.3 (build 1805)
Installation UUID: 91a3ae50-2fcb-4bbc-b185-febb51227b14
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 5.16.1.24 (d16-3 / 08809f5b)

Package version: 604000198

=== Mono Framework MDK ===

Runtime:
Mono 6.4.0.198 (2019-06/fe64a4765e6) (64-bit)
Package version: 604000198

=== NuGet ===

Version: 5.3.0.6192

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/3.0.100/Sdks
SDK Versions:
3.0.100
2.2.203
2.2.107
2.1.604
2.1.504
2.1.503
2.1.302
2.1.4
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.4.0/lib/mono/msbuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
3.0.0
2.2.5
2.2.4
2.1.13
2.1.11
2.1.9
2.1.8
2.1.7
2.1.2
2.0.5

=== Xamarin.Profiler ===

Version: 1.6.12.26
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 11.0 (14936)
Build 11A420a

=== Xamarin.Mac ===

Version: 6.2.0.42 (Visual Studio Community)
Hash: 5e8a208b
Branch: d16-3
Build date: 2019-09-17 14:19:30-0400

=== Xamarin.iOS ===

Version: 13.2.0.42 (Visual Studio Community)
Hash: 5e8a208b
Branch: d16-3
Build date: 2019-09-17 14:19:30-0400

=== Xamarin Designer ===

Version: 16.3.0.230
Hash: bbe518670
Branch: remotes/origin/d16-3-xcode11
Build date: 2019-09-17 18:39:30 UTC

=== Xamarin.Android ===

Version: 10.0.0.43 (Visual Studio Community)
Commit: xamarin-android/d16-3/8af1ca8
Android SDK: /Users/eschaefer/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
6.0 (API level 23)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.4
SDK Build Tools Version: 28.0.3

Build Information:
Mono: mono/mono/2019-06@7af64d1ebe9
Java.Interop: xamarin/java.interop/d16-3@5836f58
LibZipSharp: grendello/LibZipSharp/d16-3@71f4a94
LibZip: nih-at/libzip/rel-1-5-1@b95cf3f
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.27.1@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-3@cb41333

=== Microsoft Mobile OpenJDK ===

Java SDK: /Users/eschaefer/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 1.4.0.65
Hash: c33b107
Branch: remotes/origin/d16-3
Build date: 2019-09-19 20:42:44 UTC

=== Android Device Manager ===

Version: 1.2.0.115
Hash: 724ea69
Branch: remotes/origin/d16-3
Build date: 2019-09-19 20:43:06 UTC

=== Xamarin Inspector ===

Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 803001805
Git revision: df50511ccb08d4d84a53939f1d7fc0b40cbb0ee1
Build date: 2019-09-20 16:10:55+00
Build branch: release-8.3
Xamarin extensions: da8d448cd65f2cc87c5e6dc7e2c2ca7b96d20c72

=== Operating System ===

Mac OS X 10.14.6
Darwin 18.7.0 Darwin Kernel Version 18.7.0
Tue Aug 20 16:57:14 PDT 2019
root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

=== Enabled user installed extensions ===

Xamarin.Forms HotReload extension 1.0.0
XAML Styler 1.1.5

Log File

[nologinatgit]

I can confirm, we also experience the same.

[DevEddy]

Any infos on this? It's an update blocking issue for us.

Thank you

[radekdoulik]

Does it also happen in Debug configuration?

Do you set AndroidHttpClientHandlerType property in your project? (might be related to https://github.com/xamarin/xamarin-android/issues/3626)

[DevEddy]

Yes it does happen in Debug. No linker is involved. Where do I set AndroidHttpClientHandlerType? Is it new?

Edit: if you mean the ones from the project itself, yep. It is unchanged. Before 10.0 it worked

Edit2: I downgraded today to previous version and everything is working again as expected.

[radekdoulik]

AndroidHttpClientHandlerType is not new, it is documented here: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-process

As it is not linker related, it means it is not duplicate of https://github.com/xamarin/xamarin-android/issues/3626

[DevEddy]

Yeah I am using it ofc, it works before 10.0. I also tried different implementations to be sure - the callback is never called in 10.0.

Is there a different behaviour in 10.x or what has changed? Just try it with my repo steps, you should be able to reproduce it on your side.

Thanks again.

[radekdoulik]

@baulig could that be a bug in mono?

[nologinatgit]

As the previous version worked (and we didn't use the AndroidHttpClientHandlerType either), I guess somehow in version 10.0 the callback is ignored. I don't think that using platform specific classes is the 'solution' when a cross-platform solution was properly working in version 9.x.

[radekdoulik]

Yes, ignore the AndroidHttpClientHandlerType. I brought it up as I suspected it might be the above mentioned issue duplicate, which is not.

[nologinatgit]

WHEN you find out what is causing this, how long will a fix take to be released in a Visual Studio update? We are also about releasing a production version and this bug is preventing it.

[baulig]

This is working as expected and not a bug. You need to change your code to use the new HttpClientHandler.ServerCertificateCustomValidationCallback APIs from .NET Core.

See the API reference and API docs.

Using the ServicePointManager API (which is part of the traditional web stack and not part of .NET Core) does not make any sense with HttpClient as it would only apply to a HttpClientHandler that's based on the traditional web stack (like the .NET Framework only System.Net.Http.WebRequestHandler from System.Net.Http.WebRequest).

We are now fully compatible with .NET Core in both API and behavior.

[DevEddy]

@baulig thank you for the clarification.

Could you help me to understand before I start updating and trying it again:

1. Does it mean that the HttpClient implementation is different for Android and iOS? (As it works on iOS with the same settings).
2. I could not find those changes in the release notes, have I missed it?

PS: Can confirm that this works on Android and UWP

Thank you.

[nologinatgit]

In version 9.x I had this code:

var httpClientHandler = new HttpClientHandler();
if (!sslCheck)
  System.Net.ServicePointManager.ServerCertificateValidationCallback = (message, cert, chain, errors) => { return true; }; //no SSL check needed yet
   //httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, errors) => { return true; }; //no SSL check needed yet - NOT WORKING!

Now in version 10.x I have this:

var httpClientHandler = new HttpClientHandler();
if (!sslCheck)
  //System.Net.ServicePointManager.ServerCertificateValidationCallback = (message, cert, chain, errors) => { return true; }; //no SSL check needed yet  - NOT WORKING!
  httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, errors) => { return true; }; //no SSL check needed yet

So basically what was supposed to work in earlier versions, it is working now :)

[baulig]

At the moment, we're using the new HttpClient implementation on the following profiles:

• Desktop Mono on Linux and OS X
• Web Assembly
• Android

Check for sockets handler.sources in mcs/class/System.Net.Http:
https://github.com/mono/mono/blob/master/mcs/class/System.Net.Http/socketshandler.sources

Unfortunately, iOS is still using the old handler.

[jonpryor]

@steveisok would like to reconsider this scenario and appropriate fixes.

[EvgenyMuryshkin]

Applying fix to client instances works ok, but there are also third-party libraries, e.g. FFImageLoading, which loads images by URL using default settings from global SSL configuration. All that code is broken now.

[Cybrosys]

I thought the recommendation was to use, for iOS, NSUrlSession and not HttpClientHandler. For Android the recommendation is to use AndroidClientHandler.

Has the recommendations changed, does HttpClientHandler now support TLS 1.2+?

Why is the certificate validation callback on the HttpClientHandler if the recommendation is to not use it on iOS and Android, shouldn't the validation callback be on HttpClient instead?

What's the purpose of being able to select HttpClient implementation under project properties if there's no central way of doing custom certificate validation? From what I've read, we now have to manually create the specific HttpClientHandler we want and hook into its validation callback.

How do we do certificate validation when we have no control over the creation of the HttpClient instance, for example, when using a third-party library?

My solution, with regards to how people have used ServicePointManager, would've been to add a static callback on the HttpClient class.

[baulig]

I thought the recommendation was to use, for iOS, NSUrlSession and not HttpClientHandler. For Android the recommendation is to use AndroidClientHandler.

Has the recommendations changed, does HttpClientHandler now support TLS 1.2+?

I think you're confusing the HttpClientHandler with the TLS Provider backend that's being used internally by that handler.

Why is the certificate validation callback on the HttpClientHandler if the recommendation is to not use it on iOS and Android, shouldn't the validation callback be on HttpClient instead?

As I explained in https://github.com/mono/mono/issues/17125#issuecomment-563449127 yesterday, that's an API decision that has been made by the .NET Core team.

What's the purpose of being able to select HttpClient implementation under project properties if there's no central way of doing custom certificate validation? From what I've read, we now have to manually create the specific HttpClientHandler we want and hook into its validation callback.

How do we do certificate validation when we have no control over the creation of the HttpClient instance, for example, when using a third-party library?

My solution, with regards to how people have used ServicePointManager, would've been to add a static callback on the HttpClient class.

See my comments on https://github.com/mono/mono/issues/17125.

[mepurush]

@baulig My Xamarin.PCL project doesn't have HttpClientHandler.ServerCertificateValidateCallback. using System.Http.*

And in Xamarin.Android project the ServicePointManager.ServerCertificateValidationCallback = OnValidateCertificate;

static bool OnValidateCertificate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
var certPublicString = certificate?.GetPublicKeyString();
var keysMatch = PUBLIC_KEY == certPublicString;
return keysMatch;
}

The callback never comes.I'm using Mono.Android 10.2
How to resolve this issue.?

[DeMaxOnline]

I'm facing the same problem as @mepurush

[brendanzagaeski]

Martin can correct me if I'm mistaken, but I believe that Portable Class Libraries using the old https://www.nuget.org/packages/Microsoft.Net.Http NuGet package for HttpClient (last updated in 2015) are not compatible with custom certificate validation in Xamarin.Android 10.0 or higher.

I believe the recommendation would be to update the Portable Class Libraries to .NET Standard libraries to get the best compatibility with recent versions of Xamarin.Android. HttpClientHandler.ServerCertificateCustomValidationCallback is available in .NET Standard libraries starting in .NET Standard 1.3: https://docs.microsoft.com/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netstandard-1.3

Here are a couple links about migrating from PCL to .NET Standard:

• https://montemagno.com/how-to-convert-a-pcl-library-to-net-standard-and-keep-git-history/
• https://devblogs.microsoft.com/premier-developer/converting-pcl-portable-class-libraries-libraries-to-net-standard-class-libraries/

[mattallscripts]

So is this still an open issue then? @brendanzagaeski

[brendanzagaeski]

If my understanding is correct, then this is a closed issue. If I'm not mistaken, there wouldn't be any future plans to provide a different solution for Portable Class Libraries because Portable Class Libraries are now deprecated in favor of .NET Standard libraries:

If you’re sharing code between different .NET implementations today, you’re probably aware of Portable Class Libraries (PCLs). With the release of .NET Standard 2.0, we’re now officially deprecating PCLs and you should move your projects to .NET Standard.

https://devblogs.microsoft.com/dotnet/announcing-net-standard-2-0/

[mattallscripts]

@brendanzagaeski We have a large solution and moving our PCL to .NET Standard would be a major undertaking which we can't effort at the moment; we have multiple customers whose environments that use self-signed certs are now broken because of this upgrade which neither we nor they were expecting. Is there no alternative to moving to .NET Standard at this time?

[brendanzagaeski]

One option for a PCL project that cannot yet move to .NET Standard would be:

1. Add a class similar to the following to the Android app project:

   using System.Security.Cryptography.X509Certificates;
   
   namespace AndroidApp1
   {
       public class HttpClientHandlerCustomValidation : System.Net.Http.HttpClientHandler
       {
           public HttpClientHandlerCustomValidation() : base()
           {
               ServerCertificateCustomValidationCallback = ValidateServerCertficate;
           }
           static bool ValidateServerCertficate(object sender, X509Certificate certificate, X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
           {
               return false;
           }
       }
   }
   

2. Edit the Android app project _.csproj_ file and set the AndroidHttpClientHandlerType MSBuild property to match the custom handler type:

   <AndroidHttpClientHandlerType>AndroidApp1.HttpClientHandlerCustomValidation, AndroidApp1</AndroidHttpClientHandlerType>
   

   Replace AndroidApp1.HttpClientHandlerCustomValidation with the fully qualified name of the type, and replace the AndroidApp1 after the comma with the name of the assembly that contains the type (the app project assembly name in this example).

The ValidateServerCertficate() method will now be invoked for HttpClient requests in PCLs.

Other variations on this idea would be possible too if this option doesn't provide enough flexibility. For example, the PCL could use a technique similar to the DependencyService in Xamarin.Forms to look up the platform-specific HttpClientHandlerCustomValidation type. Then the PCL could create an instance of the type and pass it to the HttpClient constructor.

[Zambiorix]

I am seeing the same error on my Xamarin Forms app (Android), when I use ClientWebSocket ...
the solution of @brendanzagaeski does not work for me
Anyone knows how to solve that?

Thanks

[Zambiorix]

I currently solved it by installing my CA in Android emulator and on my test device..

https://lpains.net/articles/2018/install-root-ca-in-android/