Synapse: Using Twisted 19.2.0 breaks SSL verification with service-identity < 18.1

[erikjohnston]

Twisted attempts to import various modules for SSL certificate verification from the service-identity package, however if it fails to import any of those modules it falls back to its noddy inbuilt implementations that don't really work.

This breaks talking to matrix.org for push, with the error:

twisted.internet._sslverify.SimpleVerificationError: 'www.matrix.org'!='matrix.org'

The work around for those effected is to upgrade your service-identity package.

c.f. https://github.com/twisted/twisted/blob/twisted-18.9.0/src/twisted/internet/_sslverify.py#L142 and https://github.com/twisted/twisted/blob/twisted-19.2.0/src/twisted/internet/_sslverify.py#L178

[lsowen]

I was getting the same error message, and this pointed me in the correct direction. Running from service_identity.pyopenssl import verify_hostname was failing with ImportError: cannot import name 'opentype', which seems to be https://github.com/etingof/pyasn1/issues/108 (and was solved for me by https://github.com/etingof/pyasn1/issues/108#issuecomment-346659616)

[hawkowl]

The fallback made sense a while ago, but situations like this make it clear that Twisted should hard-fail if you don't have the correct dependencies. Ticket opened on Twisted: https://twistedmatrix.com/trac/ticket/9630#ticket

[richvdh]

Can we actually do anything about this without adding a hard dep on service-identity 18.1?

[erikjohnston]

This looks to by https://github.com/matrix-org/synapse/pull/5417, which bumps the minimum version of service_identity to 18.1.

[richvdh]

:tada: