Go: crypto: import fiat-crypto implementations

[mdempsky]

The fiat-crypto project (https://github.com/mit-plv/fiat-crypto) generates formally-verified, high-performance modular arithmetic implementations, useful for crypto primitives like Curve25519, Poly1305, and the NIST ECC curves that are used within the Go standard library. They're currently working on a Go backend.

BoringSSL has imported their implementations for Curve25519 and P-256: https://boringssl.googlesource.com/boringssl/+/master/third_party/fiat/

At https://go-review.googlesource.com/c/crypto/+/242177, I've uploaded a WIP CL that imports their Curve25519 implementation (w/ minor tweaks), and demonstrates a significant performance improvement over the current "generic" implementation. (The existing amd64 assembly implementation is still considerably faster though.)

This proposal is to import and make use of those implementations.

Open questions:

1. Which algorithms should be imported? BoringSSL only imports two. Should we import more?

2. Do we import both 32-bit and 64-bit implementations? We could import just one implementation and still get a performance speedup (e.g., 386 sees a -10% performance boost with curve25519_fiat_64.go, and amd64 sees a -30% boost with curve25519_fiat_32.go), but they do better still with the CPU-appropriate implementations (-30% for 386 w/ 32-bit, and -61% for amd64 w/ 64-bit).

3. How should the code be imported? E.g., should it be separated into a third_party or vendor directory with its own LICENSE file like how BoringSSL does it?

/cc @agl @FiloSottile

[mdempsky]

fiat-crypto's 64-bit P-224 implementation is about 2x as fast as Go's existing portable, constant-time P-224 implementation on amd64. Their 32-bit implementation is about the same speed on 386.

I expect P-256 will be similar to Curve25519 (i.e., existing assembly implementations are faster, but still worth measuring), but using fiat-crypto for P-384 and P-521 should be both much faster and provide a constant-time implementation (unlike the current, generic math/big code).

[mdempsky]

@agl @FiloSottile Ping.

While here, I'll point out the fiat-crypto implementation also speeds up curve25519 on ppc64le:

name               old time/op   new time/op   delta
ScalarBaseMult-32    152µs ± 1%     92µs ± 4%  -39.82%  (p=0.000 n=17+20)

name               old speed     new speed     delta
ScalarBaseMult-32  210kB/s ± 0%  347kB/s ± 2%  +65.27%  (p=0.000 n=17+17)

(I don't have any ARM workstations to readily benchmark on.)

[rsc]

I do have some concerns about adding new license notice requirements in the libraries, because those transitively apply to every Go binary anyone builds (that imports net/http at least).

I would feel much more comfortable about this if we could get the code contributed under CLA so that the Go license notice would cover it.

[mdempsky]

@JasonGross Do you think we can get fiat-crypto's Go code contributed under Google's CLA? The normal process is documented at https://golang.org/doc/contribute.html#cla.

[rsc]

Ping @JasonGross. We'd be happy to use this code but don't want to impose new notice requirements on every Go binary.

[JasonGross]

Ah, sorry, I meant to follow up on this earlier. As discussed on https://github.com/openssl/openssl/pull/12201#discussion_r472664494, MIT unfortunately doesn't permit signing CLAs on projects that it holds copyright to. :-/

[rsc]

@JasonGross, thanks for replying. I certainly understand MIT not wanting to complete CLAs.

An alternative solution to our problem of imposing new notice requirements on every Go binary would be if the generator _outputs_ could be licensed under a non-attribution license such as MIT-0 or a source-code-attribution-only license such as BSD-1-Clause.

Do you think that is a possibility?

[JasonGross]

That seems quite likely. Let me chat with me colleagues and see if it's feasible.

[rsc]

Thanks very much.

[JasonGross]

@rsc We're in the process of re-licensing under user's choice, MIT OR BSD-1-Clause. However, it seems that BSD-1-Clause is not listed under https://pkg.go.dev/license-policy, even though MIT-0 and BSD-0-Clause are. Is this an oversight? Will BSD-1-Clause in fact be sufficient?

[ianlancetaylor]

BSD-1-clause should be fine for our purposes. Thanks very much for tackling this.

I don't know why it's not listed in pkg.go.dev. Maybe it's just not very common. CC @jba .

[jba]

Looking into it.

[rsc]

BSD-1-Clause will be fine, thanks.

[rsc]

Based on the discussion above, this seems like a likely accept.

[JasonGross]

I've gotten approval from everyone and have prepared https://github.com/mit-plv/fiat-crypto/pull/881. Hopefully we'll get it merged in the next couple of days.

[JasonGross]

The code has now been relicensed under MIT OR BSD-1-Clause OR Apache-2.0

[ianlancetaylor]

Thanks!

[rsc]

Thanks so much @JasonGross!

Accepted.

[FiloSottile]

Thank you @JasonGross and everyone! Excited for this to move forward.

We can start by importing the P-384 and P-521 implementations, which currently are bad enough that they'd get the most benefit, and then move to the other fields.

The next step is figuring out a reproducible process to generate the code, and get the output clean enough to pass code review (so we are not tied forever to the code generator, if need be). Happy to help with that in the coming weeks.

[mmcloughlin]

Regarding code generation from fiat-crypto, ecckiila might be useful to look at. This was used for P-384 and P-521 in NSS 3.55.

[JasonGross]

Note, by the way, that we currently test our Rust Crate on our CI against crypto tests in Dalek, and we test our generated C code against BoringSSL tests. Currently our Go CI only checks that the code compiles, but we'd love to extend it with any sort of integration tests you think might make sense.

Note also that if you want to include the code generator in your process (rather than just fetching our checked-in generated code), our CI is probably a pretty decent place to start, as we regenerate all the code from scratch on every CI run to make sure that we're aware of any changes to the generated code.

And, of course, I'm happy to help with and support any changes you want made to the generated code itself.