Js-ipfs: 🚀 0.22.0 Release

[daviddias]

We are close to cut a 0.22.0 release that includes a ton of bug fixes and new features in IPLD land.

Highlights

✅ New API: DAG - Manipulate all your IPLD format objects

Now you can use a new DAG API to create dag-pb, dag-cbor or any IPLD Format compatible node really! Consult https://github.com/ipfs/interface-ipfs-core/tree/master/API/dag to learn how to use this new API.

✅ Test codebase refactor

We've removed a lot of duplicated and dead code from tests in js-ipfs and js-ipfs-api, tests now run faster and are more easy to write. This refactor also fixed a release bug on js-ipfs-api that would make the process run out of memory when running aegir-release.

✅ WebWorker support. 💖 @dryajov

✅ Documentation

The README was updated to make it more clear where the project roadmap lives and how the codebase is structured. You can now find a diagram for the code architecture too.

Examples were updated and polished.

✅ --hashAlg and --format support on the Block API. 💖 @kumavis

✅ Railing is now optional

Enable Railing (connecting to the bootstrapers) is now an 'experimental' feature, enable it with the BOOTSTRAP env variable. This enables you to fetch content with js-ipfs that is available on the gateways.

✅ Bug fixes

• files.get and files.add now work for recursive dirs with empty files. 💖 @pgte

Release check list

• Robustness and quality
  
  
  
  • [x] Ensure that all tests are passing, this includes:
  
  
  • [x] unit
  
  
  • [x] interop
  
  
  • [x] sharness
  
  
  • [ ] Run tests of the following projects with the new release:
  
  
  • [ ] orbitdb
  
  
  • [ ] orbit-core
  
  
  • [ ] ipfs-log
  
  
• Documentation
  
  
  
  • [x] Ensure that README.md is up to date
  
  
  • [x] Ensure that all the examples run
  
  
• Communication
  
  
  
  • [x] Create the release issue
  
  
  • [ ] Announcements (both pre-release and post-release)
  
  
  • [ ] IRC
  
  
• Remaining endeavours on track for this release
  
  
  
  • [x] Update js-ipfs and js-libp2p interdependencies to use ~ instead of ^ https://github.com/ipfs/js-ipfs/issues/755
  
  
  • [x] multiformats js-
  
  
  • [x] libp2p js-
  
  
  • [x] ipld js-
  
  
  • [x] ipfs js-
  
  
  • [x] Converge the .dag.resolve and .dag.get as described in https://github.com/ipfs/interface-ipfs-core/issues/81#issuecomment-277271941
  
  
  • [ ] Integrate remaining eth resolvers https://github.com/ipld/js-ipld-resolver/pull/71#discussion_r99754305
  
  
  • [x] Create a release checklist template that includes steps such as 'test in all of the orbit apps' https://github.com/ipfs/js-ipfs/issues/747
  
  
  • [x] _being considered_ - Complete the WebWorker support https://github.com/ipfs/js-ipfs/issues/725
  
  

What you can do to help get release done faster

Test your code against js-ipfs master and let us know if you find any hiccups?

[daviddias]

@kumavis how is the testing on https://github.com/ipld/js-ipld-resolver/pull/78 going? I'm really tempted in cutting this release and enable people to experiment the DAG API plus not have troubles installing it from npm and then releasing a patch version with all the ethereum resolvers.

Thoughts?

[daviddias]

Extra two issues to handle:

• [x] Handle https://github.com/libp2p/js-libp2p-crypto/issues/70
• [x] Merge https://github.com/ipfs/js-ipfs/pull/762

[kumavis]

eth-resolvers are ready to go

[daviddias]

Last mile for this release, Orbit testing with the new js-ipfs :)

orbit-db

Currently, not every test of orbit-db passes on master, before the link

  orbit-db replication
    two peers
Waiting for peers...
      1) replicates database of 1 entry
Waiting for peers...
      2) replicates database of 100 entries


  108 passing (2m)
  2 failing

  1) orbit-db replication two peers replicates database of 1 entry:
     Error: Timeout of 40000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.


  2) orbit-db replication two peers replicates database of 100 entries:
     Error: Timeout of 40000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

orbit-core

It fails on test:

  1) Orbit "before all" hook:
     TypeError: Cannot create property 'EXPERIMENTAL' on string './ipfs'
      at new IPFS (/Users/koruza/code/js-ipfs/src/core/index.js:22:29)
      at Promise (node_modules/ipfs-daemon/src/ipfs-node-daemon.js:40:22)
      at IpfsNodeDaemon._initDaemon (node_modules/ipfs-daemon/src/ipfs-node-daemon.js:39:12)
      at IpfsNodeDaemon._start (node_modules/ipfs-daemon/src/ipfs-node-daemon.js:24:17)
      at new IpfsNodeDaemon (node_modules/ipfs-daemon/src/ipfs-node-daemon.js:20:10)
      at Context.<anonymous> (test/orbit.test.js:49:14)

This is due to the change on IPFS creation, now when you create an instance, you have to do:

const node = new IPFS({
  repo: <repo path>
  EXPERIMENTAL: {
    pubsub: true
  }
})

@haadcode could you prepare orbit-core for the new version?

ipfs-log

Same problem as orbit-core

 Entry with js-ipfs
TypeError: Cannot create property 'EXPERIMENTAL' on string '/tmp/ipfs-log-test'
    at new IPFS (/Users/koruza/code/js-ipfs/src/core/index.js:22:29)
    at Promise (/Users/koruza/code/ipfs-log/node_modules/i

[haadcode]

Working on it

[haadcode]

@diasdavid I updated all repos for this. You should be able to run the tests with latest ipfs by doing:

cd <project>/
rm -rf node_modules/
npm i
npm link ipfs
npm test

I ran tests for all the above repos with js-ipfs from master and they all succeed except one. The one failing is the replication test in orbit-db which syncs the databases via pubsub. There's a pre-condition for the test that the peers must be connected and this is checked with ipfs.pubsub.peers(dbname) which never seem to return any peers. Perhaps something in pubsub was changed?

[kumavis]

- [ ] Integrate remaining eth resolvers ipld/js-ipld-resolver#71 (comment)

just need to bump dep ipld-resolver to 0.9.0

[daviddias]

ipfs.pubsub.peers(dbname) which never seem to return any peers. Perhaps something in pubsub was changed?

@haadcode that is an easy fix, pubsub is now behind an EXPERIMENTAL boolean as recommended by @whyrusleeping

Let me know if you have success after enabling PubSub, which is disabled by default.

As for the rest, AWESOME 👌🏽🤘🏽

[haadcode]

@diasdavid I added the flag to the IPFS init sequence as described in the doc: https://github.com/haadcode/ipfs-daemon/blob/master/src/ipfs-node-daemon.js#L40. Did I miss something?

[daviddias]

Oh, that means something is broken. What does ipfs.swarm.peers return you?

[haadcode]

What does ipfs.swarm.peers return you?

Empty array, []

[daviddias]

It means that your peers are not connected. Where is the swarm.connect call? Can you verify that it returns with success?

[haadcode]

It means that your peers are not connected. Where is the swarm.connect call? Can you verify that it returns with success?

There's no swarm.connect call. Previously, the peers always found each other automatically.

[daviddias]

Can you do a config.get of your node for me?

[daviddias]

Found the issue, you were subscribing to a topic and waiting for peers in another topic, that is why there was peers in the swarm but never in the topic pool:

  orbit-db replication
Swarm listening on /ip4/127.0.0.1/tcp/51980
Swarm listening on /ip4/192.168.2.29/tcp/51980
Swarm listening on /ip4/192.168.2.215/tcp/51980
Swarm listening on /ip4/127.0.0.1/tcp/51981
Swarm listening on /ip4/192.168.2.29/tcp/51981
Swarm listening on /ip4/192.168.2.215/tcp/51981
    two peers
GOING TO SUBSCRIBE oribt-db-tests
GOING TO SUBSCRIBE oribt-db-tests
Waiting for peers for 'oribt-db-tests2'...

Removing the + 2 makes the test successful https://github.com/haadcode/orbit-db/blob/master/test/replicate.test.js#L96

[daviddias]

Got greenlight from @haadcode ✳️

[daviddias]

o/ https://github.com/ipfs/js-ipfs/releases v0.22.0

[whyrusleeping]

:wave: "Wow" :wave: