Xamarin-android: TrustFailure exception with inner exception: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED on android and xamarin-forms on android

[xanderh]

Steps to Reproduce

(See attached project)

1. Click floating action button
2. App will attempt to enroll on our servers 10 times
3. App reports number of failures, and posts the exception to logcat

EnrollmentTest.zip

Expected Behavior

App successfully enrolls 10 times in a row, and reports no errors, every time.

Actual Behavior

App will somewhat consistently fail to enroll at first, but eventually succeed. The first attempt will almost always fail with a TrustFailure exception, but after a few tries will succeed.

The error started occurring this week out of nowhere. It's been happening to apps in development, but also published apps that haven't been updated since March.
Additionally, the error is only happening on Android, not on xamarin.iOS apps. We're seeing the issue on multiple versions of Android (confirmed on 4.4, 7.1, 8.1, and 9.0), and in apps made in xamarin.android and xamarin.forms.

The certificates for the server look fine at a glance, but I have attached the certs as they appear in logcat in case they're the problem. The certificates are the same, no matter if it succeeds or fails.

Version Information

Lastly, I apologize for not adding a label, I couldn't find a way to do so for some reason.

[radekdoulik]

Looks like duplicate of https://github.com/xamarin/xamarin-android/issues/3682#issuecomment-535834783 and https://github.com/xamarin/xamarin-android/issues/3705

Please try to use the HttpClientHandler.ServerCertificateCustomValidationCallback

[xanderh]

I'm not sure this is actually a duplicate of those issues. They seem to be about custom validation handlers not working. The problem is, we don't _have_ any custom validation handlers for SSL certificates, the built-in one worked fine up until the start of this week. Additionally, the version that worked in 9.0 but not in 10.0 still works just fine for us.
On top of that, it started happening in apps that haven't been updated since March, but we didn't see any issues before Monday of this week.

If we add the following code, the issue stops happening:
System.Net.ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicy) => { return true; };
This is obviously bad practice as it completely bypasses any validation of the SSL certificate, opening us up to man-in-the-middle attacks. We could write our own code here, but it would likely not be as secure as something coming from Microsoft.

[radekdoulik]

@baulig could you please take a look at this issue?

[Euler-KB]

Same issue here for Xamarin.Android apps built in March. Failing to connect backend

[Euler-KB]

Same issue here for Xamarin.Android apps built in March. Failing to connect backend

Ooops. Just found the cause. Was an issue with expired CA certificate (Issue might occur for all Positive SSL's purchased on Namecheap ) - Expiry date for CA cert was May 2020

https://www.namecheap.com/support/knowledgebase/article.aspx/9393/69/where-do-i-find-ssl-ca-bundle

[visuall]

Similar issue with certs from Sectigo on Android when URL which uses the cert is hit - "https://xerxes-sub.xerxessecure.com" - with httpclient. At the same time it works on iOs and Windows. Chrome also works on Droid. It is just Xamarin failing on Droid.
If we ignore the error all works but this is not good solution at all. Debugging shows incorrect certificate chain built on Android based on the article with explanations what the chain should be: https://thesslonline.com/blog/sectigo-addtrust-external-ca-root-expiring-may-30-2020. When Xamarin is debugged on ios the cert chain is correct.

[derekwheeler-tylertech]

I can confirm this is also happening with certs from Sectigo. Only on android. I also debugged the cert chain and confirmed it is terminating in the root cert "AddTrust External CA Root" that expired on 5/30/2020 rather than their cross-signed root cert "USERTrust RSA Certification Authority", (which does exist in my version of android). This article, https://www.cmu.edu/iso/service/cert-auth/addtrust.html suggests ssl client implementations based on OpenSSL v1.1.1 or before might have this issue. Is it possible the version of BoringSSL that Xamarin uses is based on an older OpenSSL fork? I would also point out that this is a serious problem that we can't fix via code, short of disabling all SSL certificate validation, which obviously we can't do. This is currently affecting at least one of our clients (an entire large US city government). Thanks

[visuall]

I have been able to go around this by using the native Android http handler - AndroidClientHandler. It is properly validating the certificate in such case.

[derekwheeler-tylertech]

Thanks @visuall . I tried that and still have the problem. I did a clean/rebuild, uninstalled the app, and rebooted the phone. But I appreciate the suggestion!
I was able to find a not-so-great solution. In the phone, if I disable the now expired "AddTrust External CA Root" root cert the problem is fixed. The problem with this solution is trying to get this information to the end users. The way to set this in the phone Settings is different in each android version and also differs from phone vendor to vendor.

[skacofonix]

I encountered a similar problem with Sectigo CA root expiration.
As @visuall say, i changed the implementation of HTTP handler from managed to Android.
This perfectly solved the problem for me

---

Implementation before
Fail to validate certificate chain with expired Sectigo cert.

public class HttpClientProvider
{
    public HttpClient Get()
    {
        return new HttpClient(new HttpClientHandler());    // Use legacy managed implementation
    }
}

---

Implementation after (Forms side)
Successfully validates the flawed certificate chain
`` public class HttpClientProvider { public HttpClient Get() { var nativeHttpMessageHandler = DependencyService.Get<INativeHttpMessageHandlerProvider>().Get(); return new HttpClient(nativeHttpMessageHandler); // nativeHttpMessageHander is injected from Android project } } ```` ```` public interface INativeHttpMessageHandlerProvider { HttpMessageHandler Get(); } ```` Implementation **after** (Android side) _Don't forget to add dependencySystem.Net.Http` from NuGet_
[assembly: Dependency(typeof(AndroidHttpMessageHandlerProvider))] namespace SmartConnect.Droid.PlatformSpecific { public class AndroidHttpMessageHandlerProvider : INativeHttpMessageHandlerProvider { public HttpMessageHandler Get() { return new AndroidClientHandler(); } } }

[derekwheeler-tylertech]

Thanks @skacofonix for those details.

For the xamarin team:

I think my problem is that I'm using the SOAP classes that the wsdl tool, (i.e. Web References->Add Web Reference), spits out in visual studio, It's not clear to me whether those use the HttpClient internally or something else. An example class that gets produced is:

//
// This source code was auto-generated by Microsoft.VSDesigner, Version 4.0.30319.42000.
//

public partial class MyExampleService : System.Web.Services.Protocols.SoapHttpClientProtocol { }

I had set the HttpClient to Android in the build settings, (https://docs.microsoft.com/en-us/xamarin/android/app-fundamentals/http-stack?tabs=windows), thinking that that setting would be used by these SOAP classes but I guess that was wishful thinking?

Thanks