Httpx: Make debugging TLS errors easier for users

[sethmlarson]

Something like this: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'momoshop.com.tw'. (_ssl.c:1108) is not user friendly. If we can somehow wrap and reraise common errors with links to documentation and a slightly more helpful message would be easier for users and keep people from turning off TLS verification.

What are everyone's thoughts on this? It's definitely a moving target and more work on our side.

[florimondmanca]

I agree that these TLS error messages are somewhat obscure. I'm not very literate about the details of SSL myself and I'm not sure what I'd need to do if I encountered the above exception.

So I think it would be great to provide information on possible causes of the exception (in plain English) and what to do as next steps (e.g. with docs links as you mentioned).

Do you already have an idea of how the custom error message could look like? :-)

[sethmlarson]

It'd be great to capture some of the common scenarios that people run into when using an HTTP library:

• Unsupported TLS version / no cipher overlap.
• Self-signed certificate
• Certificate expired
• Certificate doesn't have subjectAltName but does have commonName
• Certificate subjectAltName and hostname don't match.
• Certificate isn't in their trust store.

give the user a brief message about what happened and then point them our docs via URL.
At the URL we give a lot more detail, it could honestly just be a FAQ section of our docs that has a bunch of sub-topics of common pitfalls / errors that we can hand the user. (Cuz this could be applicable to other things too like DNS, ConnectionErrors, ProtocolErrors, etc even if we just tell a user to turn on debugging and create an issue if they can't figure it out)

Things that get people to not disable TLS or hostname verification are good, because in my experience when people hit a TLS error that they don't understand they turn off verification because at the end of the day they're just trying to make an HTTP request.

[sethmlarson]

I'm starting this work on the repo sethmlarson/tls-issues where you will feed the ssl.SSLError instance and the getpeercert() optionally into the module and it hands you back an "Issue" object with a better error message and a category. We can take the message and then generate documentation links for each of the categories like CERT_EXPIRED.

I ran HTTPS on the top 20K Alexa websites to find interesting and common issues. One of the interesting issues was that lots of sites had TLS configured properly for the www. version of their site but not the root. So I'm thinking of adding another field called alternate_hosts onto the issue to redirect to just like a browser does in the case that the original non-www host.

Thoughts?

[florimondmanca]

Sounds great @sethmlarson, looking forward to see what https://github.com/sethmlarson/tls-issues ends up like. :) Building a library to interpret SSL errors is a neat implementation idea.

[florimondmanca]

So, given the recent improvements to our exception handling, and that making dealing with SSL errors more user-friendly is a task that's pretty general to Python as a whole, so anything might as well come from a third-party package, I'm going to close this off for housekeeping purposes… But I'd still agree it'd be a _really_ nice-to-have UX-wise. Just don't see any particular action item at this point now.