Nix: Document that base32 hashes are not **the** base32 everyone expects

[piegamesde]

Is your feature request related to a problem? Please describe.
I just lost three hours (and 4 more percent of my sanity) until I figured out that base 32 hashes in Nix do their own custom foo.

Describe the solution you'd like
Please document it somewhere. No, not somewhere, but everywhere. Everywhere where "base32" is mentioned. While we're at it, we could also document which hashes are represented in which base and that no padding is used.

This includes nix-hash, nix-prefetch-url the Nix manual and all other places where one might look into when running into this issue.

[andir]

IMHO we should probably start with s/base32/nix32/g across the entire code base + some backwards compatibility in the man pages & commands such as nix-hash.

[zimbatm]

Note that base32 isn't standardized as much as base64. There are many variants out there: https://en.wikipedia.org/wiki/Base32

[piegamesde]

There's an RFC from 2006 so I think it's safe to assume at all newer applications use that one. Also I'm not sure if the nix32 algorithm is equivalent to base32 except character substitution (quick test suggests "no").

I went ahead and created a wiki page about hashes, as there are quite a few other details about them that are under-documented. Please proof-read this, as I'm still pretty unsure about a lot of the edge cases.

[colemickens]

I added links to Rust and Go implementations.

Can someone review the wiki page. Specifically the bit about SRI hashes.

@piegamesde I ask this because:

# try just prefixing "sha-256" as you documented on the wiki page
$ nix to-sri sha256-1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91
error: --- BadHash ----------------------------------------------------------------------------- nix
invalid SRI hash '1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91'

# on the other hand, if we prepend `sha256:` and call `nix to-sir` we get a **different**, valid SRI hash
nix to-sri sha256:1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91
sha256-IdU23rswdtT26QRL2e8VyMWLKfnL1K1AawWDEKVl3rw=

[edolstra]

As the Wikipedia page makes clear, there is a no standard base-32. So I don't really see a compelling reason to rename "our" base-32 after 17 years. However, nowadays it's better to use SRI hashes, which is what the new CLI defaults to.

[piegamesde]

Thanks for the additions. Also thanks for mentioning nix to-sri, I haven't seen that before and it looks something I'd want to mention in the Wiki page.

Regarding your question, it seems like the only valid encoding for SRI hashes is base64, so simply prepending sha256- is not enough.

However, now I am a bit confused about the "old <type>:<hash> format" that you used (sha256:1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91): why is there a colon instead of a dash and for which tools is this a valid input?

[edolstra]

@colemickens This is the correct command:

$ nix to-sri --type sha256 1g6ycnji10q5dd0avm6bz4lqpif82ppxjjq4x7vd8xihpgg3dm91
sha256-IdU23rswdtT26QRL2e8VyMWLKfnL1K1AawWDEKVl3rw=

The <type>:<hash> format is (AFAIK) undocumented and shouldn't be used.

[zimbatm]

To add a bit more historical context, <type>:<hash> precedes the use of SRI. It was trying to fill the same need. It mainly appears in the expected: <hash>, got: <hash> error message, which will default to SRI in the next Nix release. That format is also being used in the .narinfo files in the binary cache and maybe a bunch of other places.

[zimbatm]

Regarding Base32; rfc4648 has been published in October 2006, and Eelco's thesis in January. Technically this means that everybody else should adopt Nix's curse-proof variant of Base32 :-p