Js-ipfs: Make js-ipfs capable of working with cidv1b32

[kyledrake]

js-ipfs needs to be able to work with cidv1b32 addresses, for both creation and retrieval.

• Ability to set option in new IPFS({}) or as option to commands like ipfs add to output base32 cidv1
• Refactor the current blockstore to use multihashes for keys

Eventually we want to do a synchronized release with go-ipfs and default to cidv1b32, but for now let's start with getting the basic support in

https://github.com/ipfs/ipfs/issues/337

@alanshaw

[alanshaw]

My rough plan for this is:

1. ASAP
   
   
   
   • [x] Allow add to IPFS and get base32 CIDv1 back via an option to specify base encoding (defaults do not change)
   
   
   • [x] Retrieve content by CIDv1 key also checks for content by CIDv0 key and vice versa (compatibility)
   
   
   • [x] Default string encoding for v1 CIDs is base32
   
   
2. Next
   
   
   
   • [ ] [Migration of Blockstore to use multihash instead of CID as key](https://github.com/ipfs/js-ipfs/issues/2415)
   
   
   • [ ] DHT provider records use multihash instead of CID
   
   
   • [ ] Change defaults for adding content to CIDv1
   
   
3. Future
   
   
   
   • [ ] Ad-hoc migration - request with CIDv0 key causes copy to multihash key
   
   

For the first part of 1 there will need to be changes to multiple APIs to add a --cid-base option (or similar name - I've had no resistance to this so far) that allows the output to be encoded using base32. e.g. files.add, files.stat, files.ls, dag.put...I will post a definitive list when I have gone through them.

For the second part of 1 the local checks might be easy enough to implement. In terms of finding content that isn't local, I'd assume we'd wantlist both versions of the CID, and race them. When we get a response we'd unwant the other...? Something like that.

I will write interop tests for the second part of 1 to ensure js-ipfs and go-ipfs can get content from each other using different CID versions. I'll also PR interface-ipfs-core to document the --cid-base option in the relevant commands.

Release sync we need to sync up 2 js-ipfs/go-ipfs releases for steps 1 and 2 so that when IPFS gets these new super powers we can all still fetch content from each other. Happyness ensues and double rainbows appear.

Does this sound reasonable? Please leave your thoughts, feedback or just +1 if all good!

@whyrusleeping @Stebalien @kevina @magik6k @diasdavid @achingbrain

[richardschneider]

I don't like cid-base because its the first query string argument name (HTTP-API) with a dash (I think). There is already mhtype for multihash type. How about cbtype for cid base type,

[achingbrain]

I think --cid-base is fine, it's descriptive and fits with the --cid-version arg we already have in a few places.

[kevina]

For the go-ipfs side of (1) see https://github.com/ipfs/go-ipfs/pull/5285.

[kyledrake]

@alanshaw Is there a different meta issue for this other than this issue? Was wondering if there were any updates here this issue is missing.

[alanshaw]

No this is it. Updates are:

• https://github.com/ipfs-shipyard/js-cid-tool exists, can be installed as a standalone tool. PR for integrating with js-ipfs is here https://github.com/ipfs/js-ipfs/pull/1560
• --cid-base option PR is now here https://github.com/ipfs/js-ipfs/pull/1552 - is currently still WIP

I haven't been able to work on anything more 😢

[daviddias]

Congratz on #1560 and releasing it! :)

[alanshaw]

Update:

The CID tool was shipped with JS IPFS 0.33

JS IPFS 0.34 will include the two items from (1) ASAP:

• Allow add to IPFS and get base32 CIDv1 back via an option to specify base encoding (defaults do not change) https://github.com/ipfs/js-ipfs/pull/1552
• Retrieve content by CIDv1 key also checks for content by CIDv0 key and vice versa (compatibility) https://github.com/ipfs/js-ipfs-repo/pull/184

[alanshaw]

Tracking PRs for (2):

• [ ] https://github.com/multiformats/js-cid/pull/73
• [ ] https://github.com/ipld/js-ipld/pull/193
• [ ] https://github.com/ipld/js-ipld-bitcoin/pull/43
• [ ] https://github.com/ipld/js-ipld-dag-pb/pull/115
• [ ] https://github.com/ipld/js-ipld-dag-cbor/pull/96
• [ ] https://github.com/ipld/js-ipld-ethereum/pull/47
• [ ] https://github.com/ipld/js-ipld-git/pull/42
• [ ] https://github.com/ipld/js-ipld-raw/pull/28
• [ ] https://github.com/ipld/js-ipld-zcash/pull/35
• [ ] https://github.com/ipfs/js-ipfs-unixfs-importer/pull/21
• [ ] https://github.com/ipfs/js-ipfs-unixfs-engine/pull/235
• [ ] https://github.com/ipfs/js-ipfs-mfs/pull/38
• [ ] https://github.com/ipfs/js-ipfs/pull/1868
• [ ] https://github.com/multiformats/js-cid-tool/pull/7
• [ ] https://github.com/ipfs/js-ipfs-bitswap/pull/188
• [ ] https://github.com/ipfs/js-ipfs-block/pull/44
• [ ] https://github.com/ipfs/js-ipfs-http-response/pull/17
• [ ] https://github.com/ipfs/js-ipfs-repo/pull/192
• [ ] https://github.com/ipfs/interface-js-ipfs-core/pull/444
• [ ] https://github.com/ipfs/js-ipfs-http-client/pull/949
• [ ] https://github.com/ipfs/is-ipfs/pull/29
• [ ] https://github.com/libp2p/js-libp2p-kad-dht/pull/84

[jacobheun]

Closing this as cidv1 support has already been added. Blockstore work for multihashes is being tracked at https://github.com/ipfs/js-ipfs/issues/2415

[ribasushi]

@jacobheun It doesn't look like the global --upgrade-cidv0-in-output flag has been added to jsipfs, and I can't seem to find a ticket tracking that. Just bringing it up in case it risks slipping.