Go-ipfs: Switch to ed25519 keys by default so that we can reasonably store signed content records

[jacobheun]

To be confirmed:

• Add support for specifying key type in ipfs init
• Switch initialization to use ed25519 as the default instead of rsa

Success Criteria

• Newly created IPFS nodes will use ed25519 keys by default
• Users can rotate their existing keys to ed25519

What's Left

Mandatory:

• [x] Key rotation https://github.com/ipfs/go-ipfs/pull/7515
• [x] Sharness test bug https://github.com/ipfs/go-ipfs/pull/7542
• [x] Having IPNS keys (in the CLI, everywhere?) output base36 by default https://github.com/ipfs/go-ipfs/pull/7554

Nice to Have:

• [ ] go-multiaddr support for CIDs as PeerIDs https://github.com/libp2p/go-libp2p/issues/976
• [ ] perhaps have an option for switching peerID outputs everywhere to base36 - this can be deferred

[RubenKelevra]

Will self also be changed to ed25519?

[willscott]

Users can rotate their existing keys to ed25519

What does this mean, exactly?
It's easy enough to generate a new ed25519 key. is there / does there need to be a way to indicate that identity transition to peers? Enumerating what needs to change in a rotation would be useful.

[Stebalien]

It means #5880. That is, users need to be able to "re-key" their node without changing anything else.

[Stebalien]

@RubenKelevra yes. self _is_ the node's ID. Note: we _won't_ automatically change user keys, we're just (a) providing an easy way to do this and (b) changing the defaults for newly initialized nodes.

[RubenKelevra]

@RubenKelevra yes. self _is_ the node's ID. Note: we _won't_ automatically change user keys, we're just (a) providing an easy way to do this and (b) changing the defaults for newly initialized nodes.

Will this be backwards compatible? I mean, can for example 0.4.23 handle nodes with ed25519 keys today?

[Stebalien]

Yes.

[lidel]

When we switch to ed25519 by default, we should ensure that keys are always represented as CIDv1 in Base32Base36 with libp2p multicodec (see RFC/0001-text-peerid-cid.md).

FYSA subdomain support in our HTTP Gateway is already doing the conversion if someone uses anything other than CIDv1Base32:
https://github.com/ipfs/go-ipfs/blob/0114869d7e7d43038d1a05cd76fec22aa07f78ff/core/corehttp/hostname_test.go#L29-L30

https://github.com/ipfs/go-ipfs/blob/0114869d7e7d43038d1a05cd76fec22aa07f78ff/core/corehttp/hostname.go#L312-L343

[Stebalien]

What's the motivation?

Also, I don't believe we have support for CID-encoded peer IDs in multiaddr yet.

[lidel]

Main motivation is to make a single default change (rsa+b58 → ec+b32)
instead of two (rsa+b58 → ec+b58, and then ec+b58 → ec+b32).

Second motivation is for users to be able to copy ipns key and open it from gateway without the need for conversion step:

• https://QmPRB9SNEfYaTaZ88VLCpddTYD45B53sC8yH8xYZXGPLJp.ipns.dweb.link
  (copy-paste of b58 without conversion → breaks)
• https://dweb.link/ipns/QmPRB9SNEfYaTaZ88VLCpddTYD45B53sC8yH8xYZXGPLJp
  (converted b58 → ok)
• https://bafzbeiaqarwnhp7vmqgkhfbqyf32qgkf65xluww5vr2xlang2rekfiquku.ipns.dweb.link/
  (b32 without conversion → ok)

---

@Stebalien RFC/0001-text-peerid-cid.md is tracked in https://github.com/libp2p/specs/issues/216 but I don't see multiaddr on the list – what remains to be done?

[Stebalien]

We need to update https://github.com/multiformats/go-multiaddr/blob/8ae68cf376240f0db2decc6bed91d8c5bc198e75/transcoders.go#L293-L315 to allow CIDs. It's pretty trivial but, IIRC, the main concern was adding a dependency on CID (but that's no much of a concern).

However, I'm not sure we're ready to use this by default given that we only _just_ started supporting it this last release while we've supported ed25519 keys for quite a while. On the other hand, we could just use it as a motivation to upgrade?

[lidel]

I'd prefer if we switch both defaults together to limit the number of revolutions in PeerID land, but world would not end if we punt it until we switch default CID representation.

That being said, I believe we should solve https://github.com/ipfs/go-ipfs/issues/7318 before switching to ed25519 :(

[lidel]

Quick updates on related changes:

• JS: [email protected] library shipped with support for Base36, which solves the problem of representing ED25519 keys on subdomain gateways.
• GO: go-ipfs 0.6.0 supports conversion to DNS-safe CIDv1 in Base36:
  console $ ipfs cid format -v 1 --codec libp2p-key -b base36 z5AanNVJCxnNXk3KysfhKVHaJecKEY2RxgsrNaGb89fuzXVqjaEndN1 k51qzi5uqu5dj16qyiq0tajolkojyl9qdkr254920wxv7ghtuwcz593tp69z9m
• I've updated cid.ipfs.io and it now supports analysis of ED25519 PeerID as CIDv1 in Base36 such as k51qzi5uqu5dj16qyiq0tajolkojyl9qdkr254920wxv7ghtuwcz593tp69z9m

[lidel]

Subdomain gateway support for ED25519 being reviewed in https://github.com/ipfs/go-ipfs/pull/7441

[aschmahmann]

@lidel as mentioned in https://github.com/ipfs/go-ipfs/pull/7441/files#r447624101 it sounds like we have a decision to make about whether all libp2p keys should be represented in base36 by default.

What do you think are the inputs into this decisions? It seems like doing it is useful because if we use base32 by default then people can't copy-paste them into web browser subdomains and won't be able to do a quick visual check on them. Is the downside related mostly to people being shocked that all of a sudden their existing RSA keys are being represented as a base36 libp2p-key instead of just QmXYZ? If so how are we planning to decide yes/no?

[lidel]

What do you think are the inputs into this decisions?

It is a mixture of hard constraints and nice-to-haves.
Below is my attempt to separate them.

Constraints:

• (I believe) We need to use cidv1b36 for ed25519 everywhere to bridge the spec gap between CID and PeerID specs and make it easier for people to reason about those identifiers. The key here is using CIDv1 for ed25519 everywhere. Longer explainer below.

• We are forced to use Base36 encoding for ed25519 because of subdomains (#7441).
  Due to this, it makes sense to use Base36 as the default for those keys, avoiding the conversion step.

Nice-to-haves:

• When go-ipfs makes the switch to ed25519 by default, and those are encoded as CIDv1 in Base36, we may also switch the text representation of old RSA keys to use the same encoding. For the same of simplicity, improved UX and removal of pre-CIDv1 technical debt. But this is not a must.

Longer rationale below. (I'm sorry, it was late and I failed to make it shorter :()

---

Why we MUST use CIDv1B36 for ed25519

I think we need to recognize that switching to inlined ed25519 surfaces an edge case in our specs and introduces sever UX challenges if we keep text output as Multihash in Base58btc.

Namely, according to decoding algorithm in CID spec a CIDv0 is a Multihash in Base58btc starting with Qm. Everything else is CIDv1+.
That means an ed25519 Multihash starting with 1.. is NOT a CIDv0, and libraries will try to parse it as CIDv1+ resulting in errors.

To illustrate how bad it gets (defaults, go-ipfs 0.6):

• ipfs key gen -t ed25519 produces libp2p-key as Multihash in Base58btc (12D3KooWP3ggTJV8LGckDHc4bVyXGhEWuBskoFyE6Rn2BJBqJtpa)
• User tries to convert it to Base36 via ipfs cid command to use it in subdomain
• ipfs cid does not recognise multihashes starting with anything other than Qm.. (CID spec says Qm is CIDv0, everything else is parsed as CIDv1+):
  console $ ipfs cid format -v 1 --codec libp2p-key -b base36 12D3KooWP3ggTJV8LGckDHc4bVyXGhEWuBskoFyE6Rn2BJBqJtpa 12D3KooWP3ggTJV8LGckDHc4bVyXGhEWuBskoFyE6Rn2BJBqJtpa: selected encoding not supported
  This is go-ipfs. I think it is safe to assume this problem will occur in other places/libs too.
  
  
  
  • Ok, so user tries to inspect the libp2p-key at cid.ipfs.io/#12D3Ko.. and it looks fine. JS app at cid.ipfs.io does not follow spec to the word, and identifies it as CIDv0. There is CIDv1Base32 version at the bottom. It is cool this works, but discrepancy between CLI and web tooling does not smell well.
    
    If they copy Base32, it can be converted to Base36 just fine:
    
    console $ ipfs cid format -v 1 --codec libp2p-key -b base36 bafyaajaiaejcbrerngkbad43x2wkgyoewpfspx4dkebmw4iygvqupxmhlh5ljdy3 k51qzi5uqu5dl2yn0d6xu8q5aqa61jh8zeyixz9tsju80n15ssiyew48912c63
    
    I mean.. _not the best user experience_ ¯_(ツ)_/¯
  
  

I don't think we really have any alternative here, when we switch to ed25519 by default, we need to output them as CIDv1 in Base36.

Why we (probably) SHOULD use CIDv1B36 for all libp2p-keys

In short: devexp, simplicity, removes cognitive overhead.
People could use regular CID library for extracting libp2p-key multihash.

Hardcoding check for Qm was a thing for a long time, and now that we have had CIDs for a few years developers assume every identifier in ecosystem is a CID. Many use CID parser for Qm.. PeerIDs already, and it works only because 1.. ones are rare.

I'm mildly in favor of switching text representation of all keys to CIDv1 Base36 (including RSA):

• Representing PeerID/libp2p-key as CIDv1 will remove the surface for bugs like the one described above, and free us from that technical debt in user interfaces.
• Added value is that CIDv1 will include libp2p-key codec, making all keys self-describing, improving UX even further
• Downsides:
  
  
  
  • user may have Multihash PeerID in Base58btc hardcoded somewhere or do naive string comparison in code
    
    (I'd say it's better to break those setups sooner than later)
  
  
  • changing text representation from Base32 to Base36 in subdomain will change the Origin of IPNS websites using old RSA keys
    
    (same, I'd argue its better to do this change now that IPNS is _not yet in its prime_ and the userbase is smaller)
  
  

[aschmahmann]

Closed by #7579