Runtime: Add support for SHA3 (Keccak)

[ghost]

Since SHA1 has been known as an "unsafe" algorithm, but now a safer algorithm called "SHA3" is created. So this algorithm should also be included here. Any plans or options for Microsoft now? As far as we see, SHA3 is a little faster than SHA2 and what's more——It's SAFER.

[bartonjs]

It's been on the watch list since it was announced. Since we don't implement cryptographic algorithms within .NET we're waiting on support from the underlying platforms (Windows CNG, Apple Security.framework, and OpenSSL).

As far as speed, I'm given to understand that SHA-3 can never be as fast as SHA-2, due to an inability to hardware optimize it.

[ghost]

Hello @bartonjs：

Thanks for your quick reply, and I get the complete code from here: http://keccak.noekeon.org/KeccakReferenceAndOptimized-3.2.zip.

For more about speed and other things, please read here：http://www.drdobbs.com/security/keccak-the-new-sha-3-encryption-standard/240154037?pgno=1

[bartonjs]

Yeah, Keccak is (per your linked article) 12.5 cycles/byte. SHA-2-512 with 64-bit instructions on an Intel processor is 8.5 cycles/byte.

That said, we're just at the mercy of
1) Our underlying providers supporting it.
2) Figuring out what to name the classes. Since SHA-3 also supports a 256, 384, and 512-bit mode, we can't go with "SHA256", etc. And "SHA3256" is a bit weird.... "SHA3_256" violates our naming rules, "SHA3-256" is an invalid identifier.

[ghost]

Well……I think you can rename it as "Keccak256,384 or 512", because Keccak is just the algorithm of SHA3 as the standard one.

A persudo code may look like this following:

var keccak = Keccak256.Create().……;

Also something like for these providers:

Keccak256CrytoServiceProvider
Keccak384CrytoServiceProvider
Keccak512CrytoServiceProvider

[gvanas]

Sorry to jump in the thread, but I would suggest to consider the SHAKE extendable-output functions (XOF) from FIPS 202, or the cSHAKE from SP 800-185. They are more flexible than the plain SHA-3 hash functions and have about the same speed as SHA-2.

For higher speed, there is ParallelHash [SP 800-185] or KangarooTwelve. https://twitter.com/KeccakTeam/status/834789451708628995

Kind regards,
Gilles (a Keccak co-designer)

[JonHanna]

SHA3_256 seems a reasonable enough reason to bend the rules.

[morganbr]

Based on @gvanas 's points, I'd be interested to see an API design that keeps those other functions in mind. Maybe that would even affect the naming question.

[ghost]

@JonHanna：Agree. The name is something like the inner class that isn't publicly published to the public to be used, it's really a bit strange for us. Maybe this can be used as a wrapper or something else like this dynamically generated. And for us, the published one can be something like what I've mentioned above in my post.

[vanillajonathan]

The naming SHA3 violates the Microsoft naming guidelines which dictate that three-letter algorithms should not be uppercase.
https://docs.microsoft.com/en-us/dotnet/standard/design-guidelines/capitalization-conventions

The proper name according to the capitalization convention would be Sha3.

[jdluzen]

My port of Keccak (including modified SHA3 padding) integrates with HashAlgorithm as the others in that namespace. I would be happy to contribute it.

[gvanas]

@jdluzen Nice! Ideally, your code should also support the SHAKE and cSHAKE functions. Not much extra work, the underlying function is the same.

[morganbr]

@jdluzen, we appreciate the offer, but we have a strict policy of only using cryptographic algorithms provided by underlying platforms (CNG on Windows, OpenSSL on Linux, Apple Crypto on Mac). That ensures we don't have to worry about provable correctness or avoiding side channels. Right now, I think OpenSSL has SHA3, but not the other two platforms.

[Spacefish]

Well, we could implement the SHA3 API and just throw a "PlatformNotSupportedException" exception, when running on Mac or Windows.

[danmosemsft]

Well, we could implement the SHA3 API and just throw a "PlatformNotSupportedException" exception, when running on Mac or Windows.

@Spacefish we generally try to avoid this, as it makes it harder to write cross platform libraries if an API that seems relevant to all platforms only works on some.

[markusschaber]

As far as I can see, there are several free implementations (at least BouncyCastle and the one by @jdluzen mentioned above), so users have alternatives as long as it's not supported in the Framework.

[GrabYourPitchforks]

@bartonjs Is SHA3 realistically something that we're going to be able to support in the near-to-intermediate future? Should we resolve this issue for now and reopen it once there's a critical mass of support?

[TYoung86]

(Just a heads up; I don't feel as strongly as the wording I've written here implies, this one's a little cynical.)

You might also want to consider the impact on driving adoption in the direction _opposite_ vendor support to encourage vendor support by creating adopters by creating the API.

When SHA3 is suggested and no implementation is present the framework, at present (emphasis on _it's 2020_) many architects opt for SHA2 and concrete their architectures around it.
The lifespan of SHA2 is shortening, and as SHA3 was standardized in late 2015, given that now we're looking for post-quantum solutions like SPHINCS+ and others - the NIST began officially requesting post-quantum solutions in 2017, and now we're on round 2 of PQC standardization - the ball _appears_ dropped.

Those that do opt to use SHA3 end up using OpenSSL/BouncyCastle/etc. or rolling their own, _which is perfectly fine,_ and 'demand' is met and none is perceived by the OS vendors or framework.

OS vendor implementation appears to be driven by demand, of which dependent frameworks are the prime mover. So then .NET and Windows CNG are both waiting on each other. (Or do you think there are relatively many that use Windows CNG directly instead of as a supplment or fallback?)

Microsoft has their own PQC initiative, sans hashing algo at present. Wait much longer, the question won't be "Where's SHA3?" it'll be "Where's PQSHA1?"

Microsoft added XTS mode for AES in Windows 10, though I honestly have no idea if you can use it via System.Security.Cryptography. :shrug:

I think the general consensus of those still using SHA2 is vaguely "the world didn’t move to SHA3 because almost none of the world’s software or hardware supported it" ... which isn't true in the technical sense of software and hardware, only true in the business sense of software and hardware.
The .NET framework, Java, and other language-level frameworks are what the business sense is going to refer to. Many of the crypto-kids went to Rust, Go, JS, back to C/C++ (but not to Windows CNG, obviously) and even OCaml - but you don't _want_ to count that as demand though you probably should.

Since you're both (OS vendor and framework) waiting on demand that isn't measured by uses of the algo. in general or whether the algo. _should be used,_ or as some sort of leading example w.r.t ASP.NET Core, you may as well do the industry a favor just declare the whole namespace legacy support and encourage the use of some 3rd party solutions for legitimate up-to-date solutions or create a 1st party .NET crypto lib solution, but you _probably can't_ for some reason that involves recursion.

We'll call it the SHA3 pickle.

edit: So no Keccak256Managed/SHA3b256Managed then?

[Neustradamus]

.NET 5.0 is here!

SHA3 has been added?

Linked to:

• https://github.com/microsoft/dotnet-framework-early-access/issues/44