Go: proposal: x/crypto: Add FF1 and FF3 Format Preserving Encryption algorithms

[anitgandhi]

Introduction

I have a package written that implements the new NIST approved FF1 and FF3 Format Preserving Encryption (FPE) algorithms. You can read about FF1 and FF3 in the NIST 800-38G Recommendation.

While the algorithms are technically written to use any underlying block cipher, as of right now, only AES is approved to be used. As such, they are considered as new modes of AES.

There is a partial existing implementation however it generally only works with strings that aren't too long, as the uint64 buffers overflow; also it only implements FFX ("old" name of FF1, though the two are mathematically equivalent).

Proposal

I'm proposing that these be added to the golang.org/x/crypto package as fpe/ff1 and fpe/ff3 sub-packages.

If this proposal is accepted, I volunteer to provide the implementation.

Notes

As of right now, the way I have written my implementation, it requires Go 1.6 as it uses math/big Text function for interpreting numbers as strings of a certain base/radix. I will look into better backwards-compatible support if requested.

NIST has also provided test vectors, which I used for the test cases.

CC: @agl

[agl]

Are these needed for implementing anything that Go coders might want to implement? (The bar for x/crypto is pretty low, but things are expected to be generally usefully to at least a small number of people.)

[minux]

should we consider adding a 2nd level directory structure to x/crypto? the
top level packages are too crowded, which makes it difficult for users to
find packages.

If ff1 and ff3 are treated as a separate mode of block ciphers, perhaps we
can place them under x/crypto/mode/{ff1,ff3}? (I believe these two packages
are the first mode additions, so it's probably ok to place them under
x/crypto/mode?)

[anitgandhi]

I agree that it should at least be used/needed by others. I can say that it is certainly being used in my place of employment.

Perhaps part of the reason is that these algorithms themselves are relatively new. While their use is inherently niche, part of the reason they might not have propagated into wider scale use is because they are not as well known yet. I haven't even seen a reference implementation yet either.

I would understand if this proposal was declined for that reason :)

[anitgandhi]

Gentle ping

[minux]

I suggest the first step is to publish a go-gettable package for ff1 and
ff3. And then we can at least discuss API issues.

[rsc]

By putting this in the crypto repo we're taking on ownership and maintenance, so there is a cost that needs to be balanced with benefit to users. It sounds like there are very few users right now. Based on that, it sounds like we should decline this proposal for now (like Minux said, I'd encourage you to publish your own ff1/ff3 implementation GitHub for use with 'go get'), and then revisit if/when those become important.

[jefferai]

@rsc / @minux We get this request _quite_ often from Vault users. I think the visibility issue is that most uses of this kind of encryption are in the financial and government worlds, and they generally don't like to talk publicly about anything they're doing that is security-related.

[anitgandhi]

@jefferai and to anyone else who comes across this issue:

As I wrote the standalone/go-gettable package for my employer, we are evaluating whether or not we can open source the code. Depending on how things move forward, I will leave an update if I am able.

Thanks.

[anitgandhi]

Good news folks; we've just open sourced the library!

Feel free to take a look and review/contribute!

https://github.com/capitalone/fpe