Runtime: Security crypto - Roadmap

[karelz]

We have put together roadmap for Security crypto investments. We present it here for early feedback and further discussion in community.

The order is roughly in priority order. To see our release vehicle plans, please check individual issues on CoreFX, linked from below.

1. ASN.1 (DER) reader and writer

Tracking: https://github.com/dotnet/corefx/issues/21833

Description:

• DER is data encoding format used in PKCS and X509.
• It's hard to get right (even we had couple of iterations).

Business justification:

• We see customer ask every 2 weeks on StackOverflow (incl. viable workarounds for other missing features in crypto). Examples:
  
  
  
  • https://stackoverflow.com/questions/41808094/correctly-create-rsacryptoserviceprovider-from-public-key/41835706#41835706
  
  
  • https://stackoverflow.com/questions/41232601/uwp-ecdsp-signature/41247153#41247153
  
  
  • https://stackoverflow.com/questions/43568189/sign-an-array-of-bytes-with-x509-certificate-in-net-core/43787964#43787964
  
  
• Dependency of other features - we have internal implementation without good (public) API surface. We use it internally (a lot) on Linux and Mac.

Cost: 1-2m
Status (2018-04-01): A prototype (as internal classes) is being used in the 2.1 release. Some API/usability issues still need to be worked out.

2. Certificate Chain Pinning

Tracking: https://github.com/dotnet/corefx/issues/16364

Business justification:

• Required for multi-tenant scenarios.

Cost: 2w-1m

3. Load RSA, ECDSA, DSA keys from file

Tracking: https://github.com/dotnet/corefx/issues/20414

Description:

• Import RSA, ECDSA, DSA keys from files

Business justification:

• Native Linux scenario. Currently we have workarounds which are Windows-style.

Notes:

• We have internal implementation, without public API.
• Depends on DER feature above.

Cost: 1-2m (partial support 2w)

4. Crypto algorithm AES-GCM, AES-CCM

Tracking: https://github.com/dotnet/corefx/issues/7023

Notes: We need to tweak current crypto API design (doesn't fit these crypto algorithms inputs)

Cost: 2m

5. X509SubjectAlternativeNameExtension

Tracking: https://github.com/dotnet/corefx/issues/22068

Description:

• Expose rich APIs for certificate metadata

Business justification:

• Customers currently use incorrect/incomplete workarounds - parsings ToString() results, which is error-prone e.g. due to localization breaks

Cost: 2w-1m

6. Port crypto algorithm ECDiffieHellman from NetFx (DONE)

Tracking: https://github.com/dotnet/corefx/issues/8158

Description:

• Port crypto algorithm ECDiffieHellman from NetFX (3.5), Xamarin will benefit as well

Business justification:

• Not commonly used, but when needed, it is very important
• Blocks some migrations to .NET Core from .NET Framework

Cost: 2w-1m
Status: Available in .NET Core 2.1 (as of preview2)

7. Support messaging formats SignedCms & EnvelopedCms (DONE)

Tracking: https://github.com/dotnet/corefx/issues/14197 & https://github.com/dotnet/corefx/issues/3334

Description: Messaging formats (SignedCms, EnvelopedCms)

Business justification:

• Very popular message formats

Notes:

• We already have Windows-only implementation and Linux prototype, which hit some sort of not-roundtripping-correctly bug with our OpenSSL-based implementation on OSX.
  
  
  
  • Recommendation: Create managed implementation which is consistent across platforms
  
  
• Depends on DER feature above

Cost: 2-3m (50% for basic functionality)
Status:

• SignedCms is available to all platforms in .NET Core 2.1 (as of preview 1)
• EnvelopedCms is available to all platforms in .NET Core 2.1 (as of preview 2)
• RFC 3161 timestamps (sometimes used as CMS attributes) have API and work on all platforms in .NET Core 2.1 (as of preview2).

8. PKCS12/PFX Object Model

Tracking: https://github.com/dotnet/corefx/issues/28249

Description: An API to allow enumerating, adding, and removing items from a PKCS12 object.

Business justification:

• As a non-public API it will allow for a solution to various PFX load/save quirks on non-Windows platforms ("quirk" as compared to .NET Framework)
• As a public API it will enable customers to access SecretBag contents and to perform tasks like removing the key name property (which can remove race conditions with parallel loading of a PFX)

---

Author: @bartonjs

Suggestions, questions, additional information, or feedback on missing items are all welcome.

If anyone is interested in helping with any of the work above, please let us know. We always welcome community contributions.

[Drawaes]

Will spanifying (my new word) be seperate/tracked separately? Seems new API surfaces should be span first as byte[] has an implicit (or maybe explicit) cast or could be provided for via extension methods. Also most if not all of these methods will be cpu bound not IO so won't need async so seems a slam dunk for span?

[bartonjs]

@Drawaes Most of the new stuff will probably be Span-only (except for where it has to conform to existing interfaces / hierarchy). Adding Span to the existing crypto stuff is a parallel feature track.

[bartonjs]

@karelz Weird, I don't have edit permission here.

• Key loading: https://github.com/dotnet/corefx/issues/20414
• SubjectAlternativeName: https://github.com/dotnet/corefx/issues/22068
• ASN.1: https://github.com/dotnet/corefx/issues/21833

And it wasn't "OSX weirdness" on EnvelopedCMS for Unix, since we didn't have a Mac-specific version. But we hit some sort of not-roundtripping-correctly bug with our OpenSSL-based implementation. While I may not be a huge fan of OSX security libraries we shouldn't blame them unnecessarily :smile:.

[Drawaes]

@bartonjs, cool and understood... looking forward to it.

[vcsjones]

Port crypto algorithm ECDiffieHellman from NetFx

Is this is going to be a pure port from NetFx that uses CNG a la ECDHCng, or will this work on all platforms? CommonCrypto and OpenSSL all have the APIs to make this work. It would be very disappointing if this didn't work on all platforms initially.

As the business case says, when it's needed, it's important, and it's equally so to be able to support all platforms.

[vcsjones]

If anyone is interested in helping with any of the work above, please let us know. We always welcome community contributions.

Happy to, all of these are very much high on my list of things I would make an effort to dedicate time to.

[bartonjs]

Is this is going to be a pure port from NetFx that uses CNG a la ECDHCng, or will this work on all platforms?

• ECDiffieHellman will be ported as-is (with some possible Spanification), because I got it x-plat ready for 4.6.2 (and ECParameters added in 4.7).
• ECDiffieHellmanCng will get ported as-is.
• ECDiffieHellmanOpenSsl might make sense (I know that there are HSM scenarios for ECDSA, I don't know that there are for ECDH, so I don't know if "I opened this with a P/Invoke, here's the SafeEvpPKeyHandle" is necessary).
• ECDiffieHellman.Create() (and ECParameters/ECCurve overloads) will emit an opaque type based on CNG (Windows), Security.framework (macOS), or OpenSSL (other).

[vcsjones]

I know that there are HSM scenarios for ECDSA, I don't know that there are for ECDH, so I don't know if "I opened this with a P/Invoke, here's the SafeEvpPKeyHandle" is necessary

There is, but not in the same way we might use an HSM with ECDSA. It's more about hardware acceleration with local, specialized, hardware for fast generation since ECDH is commonly used with ephemeral keys and need to be generated quickly. They just happen to speak PKCS#11 so it's very much like an HSM. In this particular case, being able to provide a key handle that was supplied by an OpenSSL engine for said hardware would be very nice.

If it's not a stretch, I would say being able to provide a key handle is a desirable feature.

ECDiffieHellman.Create() (and ECParameters/ECCurve overloads) will emit an opaque type based on CNG (Windows), Security.framework (macOS), or OpenSSL (other).

👍

[clairernovotny]

This may be a dumb idea, but to the extent possible, can you work with or incorporate the work Bouncy Castle has already done in this space? I believe they support most of the things on this list. To the extent native code could speed certain areas up, that'd be a worthy contribution back. Basically, can .NET "adopt" BC at all? Does it have to be rewritten? They are MIT licensed.

[richlander]

@bartonjs I fixed your permission issues.

[leastprivilege]

I am certainly not a crypto expert - but I have been to some companies where BouncyCastle is banned because of implementation flaws in the past.

[karelz]

@bartonjs Thanks! I incorporated your comments in the top post.

[clairernovotny]

@leastprivilege fair enough, but surely the implementation can and should be reviewed, then fixed? Even if a new implementation is created, there's no guarantees there'll be no flaws in those either. Any codebase is subject to issues.

[karelz]

I think the key point is that we intentionally rely on OS / OpenSSL libraries to solve the crypto properly (and flawlessly). My understanding is that BouncyCastle reimplements those parts which is something we want to avoid in our code base.
I let @bartonjs to chime in with more details, but unless I am mistaken, this is the key reasoning.

[clairernovotny]

@karelz it is a full re-implementation. Where I thought of it was in the context of the ASN.1 parsing since that appears to be net-new on this list. Of course wrapping OpenSSL / OS code that does this is another good approach too. I was hoping that a new managed ASN.1 layer wasn't recreated :)

[xiaoyinl]

Do you have a plan to implement ChaCha20-Poly1305 cipher?

[vcsjones]

Do you have a plan to implement ChaCha20-Poly1305 cipher?

I think that is going to be blocked by platform support. CNG and CommonCrypto do not support CHACHA or POLY1305 yet.

[galvesribeiro]

@karelz Any chance to get DES/TDES and MAC/CMAC here? They are most used on financial and military industries and we always missed it in .Net.

Thanks

[Drawaes]

I really hope there are not any financial or military uses for this still. Seeing as DES was broken by the EFF in less than 24 hours, and that both have been withdrawn from FIPS and to my knowledge at least DES has been withdrawn from NIST standards. Implementing insecure algos in a new API doesn't seem like a good idea.

Maybe @blowdart has thoughts on that 🤣

[blowdart]

It's not up to us to limit here. Certainly we can advise, but if people need DES then they should have it. Heck, EMV uses 3DES and that's long deprecated.

[Drawaes]

:) sure but it should at least not be "obvious" or something, if something is in there for backwards compat that is okay... and I understand the issues with Obsolete. But I would hate for someone to use it in a greenfield app cause it was an "option" in there. Maybe some kind of

👊 that hits you if you try to use it?

[galvesribeiro]

Yes, DES and 3DES still PCI and DoD compliant and EMV uses it. Don't ask me why... I know it is crap old, just that it is used by 2 of the biggest industries worldwide.

[vcsjones]

I seem to recall a certain someone saying Roslyn analyzers for bad crypto practices was coming...

I would like to see some "legacy" crypto return as well. 3DES in particular.

[Drawaes]

Sure implement it with an analyser. Out of interest DoD still say it's compliant even though its been withdrawn from FIPS? I assumed (probably wrongly) that they followed FIPS. And yeah I forgot about retail banking... :)

[galvesribeiro]

Yeah... They still certify HSMs that use a few "legacy" crypto. I'm pretty sure that once the fully start rejecting such alghos, financial industry (and PCI) will drop it as well. The problem is the HUGE user/device base in the field...

[vcsjones]

and ECParameters added in 4.7

Does that mean we'll get X25519 support for free with ECDiffieHellman?

[bartonjs]

I would like to see some "legacy" crypto return as well. 3DES in particular.

Everything except RIPEMD160 and MACTripleDES are in .NET Standard 2.0 / .NET Core 2.0. Neither of those have platform support, so they aren't likely to ever reappear.

So DES/TripleDES are already back.

(Re: if ECDiffieHellmanOpenSsl needs to be public or not) If it's not a stretch, I would say being able to provide a key handle is a desirable feature.

Implementation-wise it's fairly cheap, since we already have the assembly and interop types. Plus, someone might care about curves (see next question).

Does that mean we'll get X25519 support for free with ECDiffieHellman?

If the platform supports it. Windows supports curve25519 (per https://msdn.microsoft.com/en-us/library/windows/desktop/mt632245(v=vs.85).aspx). In OpenSSL it's probably harder, since curve25519 has no OID and OpenSSL keys it by OID. (And not every distro supports every named curve anyways).

On macOS it'll be a no, since they only support secp224r1, secp256r1, secp384r1, and secp521r1.

OpenSSL (Linux or new ECDiffieHellmanOpenSsl() on macOS, I guess) would support it via explicit parameters. Even the distros which lock down to only the 4 NIST curves have (from what I've seen) left explicit parameters available.

[galvesribeiro]

So DES/TripleDES are already back.

Where? o.O

[vcsjones]

Everything except RIPEMD160 and MACTripleDES are in .NET Standard 2.0 / .NET Core 2.0. N

Ah. I'm behind on playing with the 2.0 preview bits.

Where? o.O

It's in .NET Core 2.0, which is still in preview form. So it's coming back, it's just not final, yet. https://github.com/dotnet/corefx/blob/master/src/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/TripleDES.cs

[galvesribeiro]

Ahhh ok. Thanks!

[karelz]

@vcsjones

I seem to recall a certain someone saying Roslyn analyzers for bad crypto practices was coming...

Check this out: https://github.com/dotnet/corefx/wiki/ApiCompat

[morganbr]

https://github.com/dotnet/roslyn-analyzers already includes analyzers for MD5 and SHA1. It might be worth suggesting other weak algorithms such as DES to them.

[vcsjones]

It might be worth suggesting other weak algorithms such as DES to them.

💯 yes. I didn't realize that those analyzers had already made it in.

I'm not sure where this suggestion should go or if I should file an issue in another repo.

[karelz]

@terrajobst @pjanotti how are the roslyn-analyzers related to Immo's PlatformCompat tool? Should we merge them?

[clairernovotny]

Soo...not sure if this goes here or if I should open up a separate issue to discuss this....

It would be really nice to have an extensibility point around X509Certificate2 that allows certificates backed by other crypto providers. The existing implementation of X509Certificate2 is pretty much hard-coded to use RSACryptoServiceProvider or DSACryptoServiceProvider on Windows. Not sure about the other platforms.

My use case is that I'd like a way to support Azure Key Vault-backed certificates in a clean way and that appears impossible today. There is virtually nothing on the base class that can be overridden and the extension methods GetRSAPublicKey and GetRSAPrivateKey are pretty much hard-coded to RSACng and RSACryptoServiceProvider right now.

My ideal goal is that I could create an AzureKeyVaultX509Certificate class that derives from X509Certificate2 and that end users can work with the same as existing certificates. The constructor would take a KeyVaultClient and KeyBundle so that it would know the key identifier and public key. Sign and Decrypt operations would go to the Key Vault client instead of any native API. Calls to the GetRSAPrivateKey would get a custom implementation of RSA that knows how to call into Key Vault.

Thoughts?

[bartonjs]

@onovotny A separate issue, in corefx. I don't know how possible it will be, since the existing hierarchy exposes (and accepts) native pointers. Things like SslStream end up passing that pointer to a system library, which wouldn't understand the new concept; and if they did understand it they'd have to just throw.

But how it would work / what it would look like can be discussed on a dedicated issue.

[clairernovotny]

@bartonjs done https://github.com/dotnet/corefx/issues/22437

[terrajobst]

how are the roslyn-analyzers related to Immo's PlatformCompat tool? Should we merge them?

I think they are analzyers for Roslyn itself, i.e. if you contribute code to Roslyn or if you're using Roslyn. I don't think we should merge them -- they are a very different sets. /cc @jaredpar

[Drawaes]

@bartonjs while we are on the topic, any chance of a version bump for OpenSsl to 1.1? Seems to have been pretty stable for a while now with a lot of nasty stuff (threading for one, no need to supply locks) fixed up.

[bartonjs]

@Drawaes Anyone's welcome to help make the changes in the shim to support the 1.1 API (https://github.com/dotnet/corefx/issues/9855); but we can't drop 1.0 until all of our supported distros have a 1.1 option. It's not on this list because it's not something that I think we need to have included in the next/immediate set of goals (until/unless we end up with a "oh, we can completely drop 1.0" / "we must support 1.1 because of [whatever]" and then it becomes "and we expect @bartonjs to have it done by Friday"... because everything always has to be done by Friday, y'know?).

That said, I agree that 1.1 seems better.

[Drawaes]

Friday? normally mine needs to be done by Monday morning... so therefore if it runs overtime... I could just "get it done on the weekend" :)

Moved to https://github.com/dotnet/corefx/issues/9855

[svick]

@terrajobst

I think they are analzyers for Roslyn itself, i.e. if you contribute code to Roslyn or if you're using Roslyn.

dotnet/roslyn-analyzers does include analyzers for the Roslyn API, but that's not all that's there. If you look at the document @karelz linked to, you'll see analyzers that apply to any code (many of them are ported FxCop warnings).

[clairernovotny]

Just found a fairly gaping omission in what .NET Core 2 supports -- HashAlgorithm.Create throws a PNSE there.

I need it for cases where I don't control the algorithm used:
https://github.com/onovotny/RSAKeyVaultProvider/blob/bcbc5cd4a292d7f59cd76322aa376875f3ef92de/RSAKeyVaultProvider/RSAKeyVaultProvider.cs#L55-L73

You could hard-code a switch of known-types, but that seems far more fragile.

[bartonjs]

@onovotny That would be better expressed in a corefx issue; this "issue" is about big-picture and feature planning for the next release or so.

[clairernovotny]

@bartonjs done here https://github.com/dotnet/corefx/issues/22626. Just caught me by surprise.

[SidShetye]

What about MSDN documentation of all the crypto APIs on this roadmap? My team was looking into the x509store documentation for Linux and Mac but found nothing. We're looking at non-SSL RSA and EC certificate use cases for sign/verify and encrypt/decrypt and wanted to know the store APIs, how they are backed and secured onto the local filesystem and where those files live (paths).

Is MSDN documentation considered part of individual features (hope so) or will that be covered as a separate "documentation" sprint?

[karelz]

MSDN docs update of Security topics is already on our backlog. Maybe we should add it to this roadmap for clarity - @bartonjs do you want to add it?

[bartonjs]

@sidshetye The APIs are the same on all platforms. https://github.com/dotnet/corefx/blob/master/Documentation/architecture/cross-platform-cryptography.md#x509store describes some of the cross-platform limitations, and some of the storage aspects.

We're still working out a plan for addressing .NET Core specific (and platform specific) documentation; and given that a large swathe of of our crypto docs were written before Windows Vista we're looking to give them an overhaul. Documentation for new features is certainly part of the feature, but catching up with "platform gaps from existing features" is still being worked out.

[karelz]

Updated top post:
The order is roughly in priority order. To see our release vehicle plans, please check individual issues on CoreFX, linked from below.

[clairernovotny]

@bartonjs Came across a limitation in SignedCms's API as it exists today -- there's no way to do the signing in an extensible way.

The specific example is that we need to use Key Vault's API to do the signing of the digest and return the signed bytes. What we need is the input byte array, the hash algo, some state (where we can pack in things like the key vault name, cert name, etc). We can then implement a something that calls the KV SignAsync API and returns the bytes.

[Drawaes]

Why would you want the state in there? wouldn't an instance wrap the state for you?

[morganbr]

@onovotny, would CmsSigner accepting an AsymmetricSignatureFormatter be suitable for your needs? If we did that, you could derive your own AsymmetricSignatureFormatter with whatever you need and it would be passed the hash. We might have to look at some of the other properties to ensure there's enough information to create a CMS signer (for example, it might also need to take an X509Certificate2).

[clairernovotny]

I believe an AsymetricSignatureFormatter could work here, since we could implement that in a way that calls out to Key Vault. That said, we hit the same issue as RSA that none of the overloads return Task. That forces me to do a GetAwaiter().GetResult() since Key Vault is only async. It'd be nice to add a Task<byte[]> CreateSignatureAsync(byte[]) overload.

We can provide the public X509Certificate2, that's easy.

[clairernovotny]

Looking closer, we probably don't even need our own impl of RSAPKCS1SignatureFormatter, we can use the existing one and use a custom impl of AsymmetricAlgorithm in the SetKey method.

I already have an RSA derived type (https://github.com/onovotny/RSAKeyVaultProvider/blob/master/RSAKeyVaultProvider/RSAKeyVault.cs#L8) that I can use anywhere an AsymmetricAlgorithm is used.

In terms of extension points, if we can provide a custom AsymmetricAlgorithm into CmsSigner and its corresponding X509Certificate2 with its public key, we'd be all set.

[vcsjones]

@onovotny I would like the extension point to be open enough that signing operations are not tied to RSA. ECDSA has merit to support. I think AsymmetricSignatureFormatter is more appropriate since it can give better flexibility. Even within RSA, someone may want to use PSS instead of PKCS1.5.

[karelz]

Please let's take specific API discussions into specific issues on CoreFX (for SignedCms it is https://github.com/dotnet/corefx/issues/14197). This issue was meant for higher-level discussion on larger gaps in the roadmap.

[SidShetye]

What about Web Cryptography (JWS, JWE and JWK)? It’s gaining popularity when building modern cloud apps (e.g. Azure Key Vault, Cloud HSMs and even at Crypteron) but .NET support, even with 3rd party nuget packages, is sparse.

[loic-sharma]

It'd be nice if .NET's X509Certificate2 revocation checking APIs were fleshed out further. Current .NET APIs do not support OCSP and expose very little information about the revocation statuses. For example, there's no way to get basic CRL fields like a certificate's revocationDate.

[bartonjs]

For the 2.1 release we increased the priority of the ECDiffieHellman, SignedCms, and EnvelopedCms (since the lack of those types was an impediment to moving code to .NET Core). So with the 2.1 release the roadmap status is

• ASN.1/DER reader and writer
  
  
  
  • "Aggressively prototyped". Reader and writer functionality were written (as non-public API). The reader and writer power a prototype serializer/deserializer. That serializer powers the .NET Core implementation of SignedCms.
  
  
  • Remaining work
  
  
  • API review on the reader and writer for final shape analysis.
  
  
  • Decide if the serializer works the way we want it to (we've found some usability issues that may require a redesign), API review the serializer in its final form.
  
  
• Certificate Chain Pinning
  
  
  
  • No update
  
  
• Load RSA, DSA, ECDSA keys from file
  
  
  
  • No update
  
  
• AES-GCM/AES-CCM
  
  
  
  • A lot of active discussion on the feature, but still haven't landed on an API design.
  
  
  • Apple doesn't seem to have added public API for this yet, so still have to figure out GCM/CCM on macOS.
  
  
• X509SubjectAlternativeNameExtension
  
  
  
  • No update
  
  
• ECDiffieHellman
  
  
  
  • Will be available in .NET Core 2.1 preview 2
  
  
• SignedCms
  
  
  
  • SignedCms is available in .NET Core 2.1 preview 1
  
  
  • Bonus: Support for RFC 3161 cryptographic timestamps will be available in .NET Core 2.1 preview 2
  
  
• EnvelopedCms
  
  
  
  • Will works on macOS and Linux in .NET Core 2.1 preview 2
  
  

Since some things are popping off the roadmap other things are being pushed on:

• API to work with PKCS12/PFX objects as containers, instead of just opaque blobs.
• Some API enhancements to SignedCms (use Span/Memory where appropriate, and a couple of smallish feature requests)

[msftbot[bot]]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @jeffhandley
See info in area-owners.md if you want to be subscribed.