Cryptography: Adding type hinting for cryptography

[software-opal]

I'm using your library in a code base that uses typechecking and I'd like to be able to perform the type checking on my usage of cryptography.

One way is to add typed stubs of cryptography into typeshed.

[reaperhulk]

We talked with someone at PyCon US this year about type stubs and we decided typeshed was a good place for them to live for now. We're happy for someone to do it, but we have no plans to do it ourselves at this time.

[reaperhulk]

I looked at this a bit over the weekend and progress is effectively blocked on https://github.com/python/mypy/issues/2922.

cryptography makes heavy use of ABC registration so not supporting that is a challenge. If we wanted to do this we would need to either switch all abc registration to subclassing or write a protocol (https://www.python.org/dev/peps/pep-0544/) to match every abc interface we've defined and keep them in sync.

Neither of these options is very appealing. We could also bypass the problem by just defining these as Any, but I don't want to ship type hints that have only patchy coverage of the arguments to functions and methods.

[jlaine]

If the plan is to provide type annotations in the form of .pyi files, does it matter if the .pyi files "pretend" that we are using subclassing instead of ABC registration?

I have started putting together (partial) stubs to be able to use mypy on one of my projects: https://github.com/aiortc/aioquic/tree/master/stubs/cryptography

[reaperhulk]

With pyi files the problem is that we have to duplicate every ABC with an equivalent protocol and make sure we keep them in sync, correct?

[jlaine]

I believe so, yes

[glyph]

Another way to go about this might be to simply replace the ABCs with Protocols completely. Protocols can be defined as normal, or as @runtime protocols, which means issubclass works by doing a structural subtyping check. This should be mostly compatible with the existing interface declarations:

• you can still subclass them to get concrete implementations
• isinstance will still work, if they're decorated with @runtime
• typing_extensions does support python 2
• you can have a class decorator that insists on compliance with an interface (I described this a while ago, here: https://github.com/python/mypy/issues/4717#issuecomment-454609539)

The one problem is that I don't know of any way to give a Protocol a literal concrete .register classmethod without it thereby insisting that every adherent thereto also have a .register classmethod.

To maintain compatibility, you could define the classmethod in an if not TYPE_CHECKING block, which would define it only at runtime; it would not type-check for external libraries to call .register, but if they were using register_interface that could be appropriately modified to make it do what it needs to do. And you could get rid of verify_interface and just let Protocol's existing subclasshook do the job.

[alex]

I don't think we care about 3rd parties registering implementations of our ABCs, so changing how that works seems fine.

[glyph]

Okay, in that case I guess we're almost to the point where I could try my hand at a PR...

[jlaine]

FYI I have submitted a PR against typeshed to expand the cryptography typings: https://github.com/python/typeshed/pull/3307

[jlaine]

@reaperhulk and @alex the typings I submitted to typeshed are almost complete. Only the x509 module needs a little more love to fully support all extension types. Could you possibly weigh in on whether the PR against typeshed is going the way you want?

I will try running type checks against certbot, any other big cryptography users you can think of?

[jlaine]

mypy 0.750 just shipped, and it includes the type stubs I submitted to typeshed, any improvements are welcome.

[reaperhulk]

Since these are going into typeshed already I'm going to close this for now. Once we drop 2.7 support we'll be more capable of directly supporting this in a way that isn't insanely frustrating.