Ring: Support BLAKE2

[DemiMarie]

Blake2 is much faster than SHA-2 in software and is immune to length extension attacks: the "obvious" way of creating a MAC with Blake2 is in fact secure.

libsodium (MIT license) has a good implementation.

[briansmith]

libsodium (MIT license) has a good implementation.

A small correction: libsodium uses the same ISC license that _ring_ and BoringSSL prefer: https://github.com/jedisct1/libsodium#license.

[briansmith]

Blake2 is much faster than SHA-2 in software and is immune to length extension attacks: the "obvious" way of creating a MAC with Blake2 is in fact secure.

Which protocols are using Blake2?

I would like to have a version of Blake2 that is optimized specifically for use in Argon2, since I see that as the primary usage of Blake2 in the near term.

[pyfisch]

Why should Blake2 get added to _ring_

Blake2 is faster than the common cryptographic hash functions but as secure as SHA-3. There are [many existing uses] of Blake2 including the Noise protocol (used by WhatsApp) and libsodium. It is also the base for the Argon2 password hashing scheme. It can also be used for MAC.

Proposed API

Without native MAC mode it can use the ring::digest API. Section 4 of RFC 7693 proposes 8 standard parameter sets:

            Algorithm     | Target | Collision | Hash | Hash ASN.1 |
               Identifier |  Arch  |  Security |  nn  | OID Suffix |
           ---------------+--------+-----------+------+------------+
            id-blake2b160 | 64-bit |   2**80   |  20  |   x.1.5    |
            id-blake2b256 | 64-bit |   2**128  |  32  |   x.1.8    |
            id-blake2b384 | 64-bit |   2**192  |  48  |   x.1.12   |
            id-blake2b512 | 64-bit |   2**256  |  64  |   x.1.16   |
           ---------------+--------+-----------+------+------------+
            id-blake2s128 | 32-bit |   2**64   |  16  |   x.2.4    |
            id-blake2s160 | 32-bit |   2**80   |  20  |   x.2.5    |
            id-blake2s224 | 32-bit |   2**112  |  28  |   x.2.7    |
            id-blake2s256 | 32-bit |   2**128  |  32  |   x.2.8    |
           ---------------+--------+-----------+------+------------+

OpenSSL only supports blake2b512 (NID: 1056) and blake2s256 (NID: 1057). The shorter variants use a different initial state and are truncated in the end. They can be supported by the current API.

Existing Implementations

• OpenSSL: https://github.com/openssl/openssl/tree/master/crypto/blake2
• libsodium: https://github.com/jedisct1/libsodium/tree/master/src/libsodium/crypto_generichash/blake2 [edit: fixed URL - Brian]
• rust-blake2 by ebfe: https://github.com/ebfe/rust-blake2
• blake2-rfc by cesarb: https://github.com/cesarb/blake2-rfc
• rust-crypto: https://github.com/DaGenix/rust-crypto/blob/master/src/blake2b.rs

I have also written my own toy implementation in Rust.

[briansmith]

Blake2 is faster than the common cryptographic hash functions

We should look at the benchmarks, especially benchmarks on ARMv8 and Intel Skylake, which have accelerated SHA-2.

the Noise protocol (used by WhatsApp)

WhatsApp doesn't use BLAKE2, if I understand their documentation correctly. WhatsApp uses "Curve25519, AES-GCM, and SHA256" in its Noise Pipes implementation, and HMAC-SHA-256 and SHA-512 elsewhere, according to the Whatsapp Security Whitepaper.

It is also the base for the Argon2 password hashing scheme.

Do we think one can build a good implementation of Argon2 on top of a fully-separated implementation of BLAKE2? It seems more likely that a good Argon2 implementation requires the BLAKE2 code to be inlined into the rest of Argon2, no?

Here are the results of my Twitter poll (which, I admit, are not very useful): https://twitter.com/BRIAN_____/status/747182036939485185

I guess I find the motivation to be pretty weak, especially when measured against the goals I have for this project to be simple and to avoid adding unnecessary options. So, honestly, I'm not quite sure what to do yet.

[DemiMarie]

In my mind, Blake2 has these advantages:

• It is fast — faster than MD5 and much faster than SHA2. That alone is
  enough to justify its use in applications that aren't constrained by
  regulation or backwards compatibility.
• It doesn't need the HMAC construction because it is not vulnerable to
  length extension attacks. This is one less way for users to screw up.
• It has built-in support for keying and personalization. The keyed
  version is a secure MAC, and the personalization string can be used for
  easy domain separation.
• It has built-in support for tree hashing and parallel hashing. More
  performance wins.

[briansmith]

It is fast — faster than MD5 and much faster than SHA2. That alone is
enough to justify its use in applications that aren't constrained by
regulation or backwards compatibility.

To say it is fast is meaningless without considering implementation issues. If you have an AVX2- or Intel-SHA-Extensions- or ARMv8- accelerated SHA-2 implementation, is a naive implementation of BLAKE2 still faster? One thing that's becoming clear from the discussion is that it doesn't make sense to have a slow implementation of BLAKE2. Do we have an idea what's the _fastest_ existing implementation? If nothing else, then we can compare it head-to-head against the SHA-2 implementation that's already in _ring_?

Here's the BLAKE2's team's list of BLAKE2 users: https://blake2.net/#us

I don't think BLAKE2 is bad. I do think that presenting too many choices is bad. Would adding BLAKE2 as a choice to _ring_ be adding a useful choice, or would it be pandering to people who want to build software using 100% algorithms not approved by NIST, or something else? Is being too conservative with what we put in _ring_ going to make it people doing important and innovative stuff avoid using and helping the _ring_ project? These are the kinds of issues that I'm struggling with.

I guess the question I have, short term, is this: Is somebody building useful software that requires BLAKE2 that wants to use _ring_ and would strongly prefer that _ring_ supply the BLAKE2 implementation? If so, I'd like to hear from them, either here, or privately ([email protected]).

[briansmith]

As far as keyed or personalized hashes go: I think that we could/should, at least, rename the ring::hmac submodule to ring::mac in expectation that, at least eventually, we'll probably add some non-HMAC mechanism like BLAKE2 or keyed SHA-3.

[sneves]

If I'm not mistaken, the Skylake timings available here are already using an OpenSSL using the Intel SHA instructions (OpenSSL 1.0.2d). Neither SHA-1 nor SHA-256 are faster than BLAKE2b there. Furthermore, a pure C implementation of BLAKE2b (regs) is only 25% slower than the AVX2 code.

I don't have the hardware (Skylake or ARMv8) to confirm or deny these figures by myself. As for ARMv8, I have no idea what way the numbers go, and it's unlikely I'll find out any time soon. ARMv8 seems like an entirely developer-hostile platform.

As a separate comment, I find it highly distasteful for chipmakers to try and dictate what crypto gets used by introducing their own implementations. This results in bad primitives, like SHA-1 and the side-channel-happy small-block AES, being etched in silicon and thus much harder to deprecate when the time comes. I'd rather not incentivize this behavior, even if it comes with a small performance cost.

[DemiMarie]

I can say that Sodium's Argon2 implementation uses its Blake2b implementation as a subroutine. Therefore, if _ring_ wants to provide Argon2 (which seems likely to me, since it is the PHC winner), it won't be _much_ more code to offer Blake2b as well.

Sodium's Blake2b implementation is very fast, including versions with various SIMD optimizations. Andrew Moon also has written an optimized Blake2b implementation.

[zookozcash]

Disclosure: I'm a co-author of BLAKE2. So is @sneves.

I've seen two questions from @briansmith:

• performance details on specific real hardware; I'll defer to @sneves (https://github.com/briansmith/ring/issues/169#issuecomment-229208436) and @DemiMarie.
• Do some ring users require it? I doubt it! It is used in quite a few codebases (as @briansmith noted here), but I doubt any of those are emitting data that a new Rust codebase needs to read in, at least not yet. Maybe in the future new Rust code bases will want to interoperate with librsync, Argon2, libsodium, or Pcompress.

By the way, @briansmith, I sympathize with your desire to minimize options. I believe that options and complexity are a source of usability problems for programmers, and of security bugs.

My experience in using both SHA-2 and BLAKE2 in the Tahoe-LAFS and Zcash protocols (https://z.cash/blog/fixing-zcash-vulns.html) is that the added high-level features in BLAKE2 (compared to SHA-2) make a real difference in terms of usability, and therefore security.

For example, programmers are always building a "poor man's MAC" by prefixing a key to a message. Cryptographers know that this is vulnerable (because of length-extension), but programmers don't, so they keep doing it and they keep deploying vulnerable code. By the time of the SHA-3 contest (circa 2007), cryptographers decided that enough was enough and we needed to give programmers a hash function that would be secure even when used in this way, which is why SHA-3 required all candidates to be invulnerable to length-extension. If you're offering SHA-2 as a default option to your programmers, its vulnerability to length-extension is a danger to them. (Poor-man's-MAC isn't the only danger posed by length-extension.)

Likewise, the prudent way to use a secure hash function is with a "personalization" string that forces each different use of the hash function — both within your own protocol and also with other people's protocols — to have different outputs for identical inputs. This avoids a class of "replay-like" attacks which, in their most general form, were termed "Chosen Protocol Attacks" by Kelsey and Schneier. Almost no programmers ever do this. With SHA-2, they would have to build it on top of the hash function themselves (that's what we did in the Tahoe-LAFS project, years before the SHA-3 contest and BLAKE2). With BLAKE2, they can use the built-in API for it, which will hopefully make it more discoverable and usable to them.

It was, I assume, reasoning like this which led the authors of libsodium to make BLAKE2 the default "generic hash" — the safest choice for the user who hasn't studied the issues and is just going to use the default.

Here are my slides from when I presented BLAKE2 at ACNS: https://blake2.net/acns/slides.html

The performance numbers have improved, on all platforms, since I wrote those slides, so skip that slide.

[DemiMarie]

I haven't run benchmarks on Blake2b myself. I have followed Sodium
development, though, and (as an application programmer, not a
cryptographer) I can say that I agree with what @zookozcash said regarding
usability. I am working on a simple (not yet implemented) protocol for
communication between processes that share a secret key, and Blake2b means
that a very simple protocol (Alice sends nonce, Bob replies with nonce and
MAC of nonces with personalization A, Alice replies with MAC of nonces with
personalization B, use MAC of nonces with personalization C for AEAD) Just
Works.

[briansmith]

I think I may have misunderstood why so many people are commenting on this issue. I had thought that the people commenting were actually users of _ring_ or planning to use _ring_. But, I think instead people just came here from Twitter.

Right now, libb2 and rust-crypto both support BLAKE2 and also it's easy to make a simple BLAKE2 implementation for prototyping. So, I think people can build prototypes of stuff already using BLAKE2 without _ring_ adding it. People can also submit PRs to _ring_ that add BLAKE2 and the tests analogous to the existing SHA-1/SHA-2 tests. Then people could build stuff on top of one of those options.

If somebody builds something cool & useful on top of _ring_ and points me to some code in their project that would benefit in a non-trivial way from _ring_ adding BLAKE2, then that's a compelling enough reason for me to do work to merge a PR adding BLAKE2. But, I don't want to add dead code. In other words, the standard for adding BLAKE2 is the same as all the other functionality in _ring_, which all is being used. For example, I was going to ship the first release without RSA signing support, but people showed stuff that needed it, so their PR for it got merged.

For example, it should be no problem to build a prototype Noise implementation right now that implements X25519 + SHA-2 + AES-GCM or X25519 + SHA-2 + ChaCha20-Poly1305 without any additional work on _ring_'s part (though maybe adding support for multi-use X25519 keys would be useful for some modes). And, people can add X25519 + BLAKE2 + ChaCha20-Poly1305 using _ring_ + something else for BLAKE2. Then we could see if there's a benefit to anybody to actually add BLAKE2.

[briansmith]

I removed the good-first-bug tag as it turned out to be a terrible first bug as evidenced by the discussion above about whether or not to even do it. I suspect some _ring_ user will eventually want BLAKE2 and then it will probably become a good first bug again.

[zookozcash]

I think I may have misunderstood why so many people are commenting on this issue. I had thought that the people commenting were actually users of ring or planning to use ring. But, I think instead people just came here from Twitter.

I'll remember to mention why I'm here next time I join someone else's dev discussion. I'm a potential future user of _ring_ but not a current one.

[zookozcash]

BTW, Noise protocol supports both SHA2 and BLAKE2. See the Rationale in https://noiseprotocol.org/noise.html.

[briansmith]

BTW, Noise protocol supports both SHA2 and BLAKE2. See the Rationale in https://noiseprotocol.org/noise.html.

I am familiar with it. First, let's distinguish between what works and what is recommended. Almost any hash function, SHA-2, SHA-3, BLAKE2, Simpira, etc. works for Noise. The people who make the Noise protocol spec have chosen to endorse two as of now. Trevor also gave a reason why somebody might prefer BLAKE2 on the modern crypto mailing list. I wouldn't be surprised to see SHA-3 and Simpira implementations of Noise-based protocols soon too.

Please don't take this personally, but "I might maybe be thinking about using _ring_" isn't very motivating to me either. It's more useful to actually see people sharing real-world code that they clearly intend to deploy to production, that does something useful, when considering feature requests. Then it's clear that they're seriously invested in helping to maintain the feature they asked for.

To be clear, if Shay Gueron asked me to add Simpira to _ring_, I would hesitate the same way, because nobody _using ring_ is using Simpira now. In fact, I'm hesitating to even add SHA-3, for the same reasons.

[zookozcash]

That's fine. I wasn't trying to persuade you to add BLAKE2 support to
_ring_ just because I might use it in the future!

I was just trying to state "why I am here":

• Zaki Manian drew my attention to this issue, on twitter,
• I like to promote BLAKE2 because I hope it will become a de facto
  standard and thus promote data compatibility,
• I wanted to help by sharing relevant knowledge that I have,
• and lastly I'm interested in launching new crypto app codebases in Rust
  in the future.

I hope my contribution to this ticket hasn't impeded your work.

BTW, Noise protocol supports both SHA2 and BLAKE2. See the Rationale in
https://noiseprotocol.org/noise.html.

I am familiar with it. First, let's distinguish between what works and what
is recommended. Almost any hash function, SHA-2, SHA-3, BLAKE2, Simpira,
etc. works for Noise. The people who make the Noise protocol spec have
chosen to endorse two as of now. Trevor also gave a reason why somebody
might prefer BLAKE2 on the modern crypto mailing list. I wouldn't be
surprised to see SHA-3 and Simpira implementations of Noise-based protocols
soon too.

Please don't take this personally, but "I might maybe be thinking about
using _ring_" isn't very motivating to me either. It's more useful to
actually see people sharing real-world code that they clearly intend to
deploy to production, that does something useful, when considering feature
requests. Then it's clear that they're seriously invested in helping to
maintain the feature they asked for.

To be clear, if Shay Gueron asked me to add Simpira to _ring_, I would
hesitate the same way, because nobody _using ring_ is using Simpira now. In
fact, I'm hesitating to even add SHA-3, for the same reasons.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/briansmith/ring/issues/169#issuecomment-230134347, or mute
the thread
https://github.com/notifications/unsubscribe/APp9QOFZ2jOhypAbFNCI7R8lxyLVD25Fks5qRzZRgaJpZM4IWyqg
.

[briansmith]

No, nobody's impeding our work here. Actually, this conversation has been good because I've had to develop some policies for adding features to _ring_, which I didn't need to have before. And, I did learn that I should at least learn more about the performance issues; my prior understanding was that the Intel SHA-2 hardware implementation was going to be much faster than any software implementation of any secure hash function, but it seems I was likely wrong. So, thanks for the correction there.

[briansmith]

I just saw this: https://github.com/sneves/blake2-avx2, which claims:

• 3.19 cycles per byte on Haswell for BLAKE2b;
• 1.45 cycles per byte on Haswell for BLAKE2bp;
• 1.56 cycles per byte on Haswell for BLAKE2sp.

If/when we add BLAKE2 to _ring_ we should probably just avoid adding any x86-64 implementation that's slower than that one, except for a pure Rust fallback implementation.

[sneves]

For what it's worth, I only commented because someone asked about performance. I don't care where or why BLAKE2 gets included.

While I was right about OpenSSL having SHA instruction support, I have just discovered, with an unfortunate illegal instruction exception, that Skylake does not have those. In fact, according to this SHA instructions will only be present in Cannonlake (and according to Wikipedia, Goldmont). So it will be a while before we even know what their performance looks like.

[briansmith]

The BLAKE2X proposal says "BLAKE2X’s extract-then-expand scheme is similar (but not identical) to
that of HKDF." Is there a clear description of the difference and why there is a difference?

[zookozcash]

I'm not aware of such a document.

On Oct 19, 2016 02:44, "Brian Smith" [email protected] wrote:

The BLAKE2X proposal says "BLAKE2X’s extract-then-expand scheme is similar
(but not identical) to
that of HKDF." Is there a clear description of the difference and why
there is a difference?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/briansmith/ring/issues/169#issuecomment-254751514,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APp9QMGsWiID7RSlNggj8H4CEtmEkI2Nks5q1dhqgaJpZM4IWyqg
.

[sneves]

HKDF is based on HMAC; BLAKE2X is not, keying the hash directly. I suppose that's the main difference. Apart from that, BLAKE2X's output mode is counter-based, whereas HKDF's expansion is sequential, using the ith output block as input to the (i-1)th output block. There are other differences, mostly related to signaling things, but are relatively minor.