Js-ipfs: "Error: Invalid block" when combining `[email protected]` and `[email protected]`

[aphelionz]

• Version: 0.43.0
• Platform: Ubuntu Linux locally, Codesandbox supplied below
• Subsystem: Init

Type: Bug

Severity: Critical, when combined with the named dependencies:

"ipfs": "~0.43.0",
"ipfsd-ctl": "^4.0.1",

Description:

I'm working on the orbit-db-test-utls which creates a number of proc and go nodes via ipfsd-ctl. When creating the proc nodes, it throws an Error: invalid block. Here is the trace:

     Error: invalid block
      at Object.put (node_modules/ipfs-repo/src/blockstore.js:76:15)
      at BlockService.put (node_modules/ipfs-block-service/src/index.js:63:32)
      at IPLDResolver.put (node_modules/ipld/src/index.js:196:21)
      at async Object.put (node_modules/ipfs/src/core/components/dag/put.js:48:19)
      at async Proxy.init (node_modules/ipfs/src/core/components/init.js:128:27)
      at async InProc.init (node_modules/ipfsd-ctl/src/ipfsd-in-proc.js:93:5)
      at async Factory.spawn (node_modules/ipfsd-ctl/src/factory.js:141:7)
      at async Promise.all (index 1)
      at async Factory.factory.constructor.localSwarm (index.js:5:692)
      at async Context.<anonymous> (test/workflow.spec.js:91:21)

I believe it's related to trying to create the empty IPLD directory here: https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/components/init.js#L128

Steps to reproduce the error:

I created a codesandbox that reproduces the issue here: https://codesandbox.io/s/invalid-block-demo-d4oty?file=/src/index.js. Hope that makes it easy to see! It's also nice because I use npm locally and codesandbox uses yarn, so that can be ruled out as well.

[aphelionz]

Edited the above to remove go-ipfs-dep as the culprit and also added the commented out code block to the codesandbox to rule out a naked IPFS.create with the options supplied by the factory.

// Confirmed that the error seems to be with the combination of `ipfsd-ctl`
// and `js-ipfs`. If you comment out the above and uncomment below, You
// can see that I'm trying to create a  js-ipfs node with the same
// options as supplied by the factory
(async () => {
  const IPFS = require("ipfs");

  const ipfsConfig = {
    API: {
      HTTPHeaders: {
        "Access-Control-Allow-Origin": ["*"],
        "Access-Control-Allow-Methods": ["*"]
      }
    },
    Addresses: {
      Swarm: ["/ip4/127.0.0.1/tcp/0"],
      API: "/ip4/127.0.0.1/tcp/0",
      Gateway: "/ip4/127.0.0.1/tcp/0"
    }
  };

  const ipfs = await IPFS.create(ipfsConfig);
  console.log(ipfs);
})();

[oed]

I can recreate this issue using only js-ipfs:

$ node
> let IPFS = require('ipfs')
undefined
> let ipfs = IPFS.create({"repo": 'ipfstest'})
undefined
> generating 2048-bit RSA keypair...
(node:53224) UnhandledPromiseRejectionWarning: Error: invalid block
    at Object.put (/Users/oed/ceramic/js-ceramic/node_modules/ipfs-repo/src/blockstore.js:76:15)
    at BlockService.put (/Users/oed/ceramic/js-ceramic/node_modules/ipfs-block-service/src/index.js:63:32)
    at IPLDResolver.put (/Users/oed/ceramic/js-ceramic/node_modules/ipld/src/index.js:196:21)
(node:53224) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:53224) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

If the repo doesn't exist this will throw this error. If I run this code again the error will not appear.

Version: 0.43.0

Edit:
This looks like a very critical issue. It's affecting js-ipfs 0.40.0 as well, so downgrading is not an option.

[oed]

Looks like this might be an issue resulting from a version mismatch of ipld-block vs ipfs-block (the package was renamed, and moved to ipld org).
Releases here: https://github.com/ipld/js-ipld-block/releases
the IPLDResolver is using ipld-block (v0.9.1) while ipfs-repo is using ipfs-block (v0.8.1). See error above.

Looks like https://www.npmjs.com/package/ipfs-block is still stuck on v0.8.1 while https://www.npmjs.com/package/ipld-block is on v0.9.1. Seems like the easiest solve might be to update ipfs-block to match ipld-block?

cc @vmx

[oed]

Downgrading ipld to 0.25.3 seem to work around the issue. Problem looks to be that 0.25.4 included the breaking change to ipld-block and it got pulled into js-ipfs everywhere.

https://github.com/ipld/js-ipld/releases

[oed]

The package missmatch (ipld is the only package using ipld-block):

$ npm list ipfs-block
@ceramicnetwork/[email protected] /Users/oed/ceramic/js-ceramic/packages/ceramic-core
└─┬ [email protected]
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected]  deduped
  ├─┬ [email protected]
  │ └── [email protected]  deduped
  ├── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]  deduped
  └─┬ [email protected]
    └── [email protected]  deduped


 $ npm list ipld-block
@ceramicnetwork/[email protected] /Users/oed/ceramic/js-ceramic/packages/ceramic-core
└─┬ [email protected]
  └── [email protected]

Seems like the most immediate fix should be to release a hotfix for ipld that downgrades to ipfs-block. After that there should be a coordinated release with the breaking change.

[vmx]

That should be fixed now with the js-ipld 0.25.5 release. Thanks @oed for the analysis.

[oed]

Thanks for the fix @vmx! Glad I could be helpful :)

[aphelionz]

This appears to be fixed for me, and confirmed off-band with @oed. Closing. Thanks for your quick attention @vmx

[achingbrain]

After that there should be a coordinated release with the breaking change.

@hugomrdias is currently updating all of our modules with ipfs-block -> ipld-block, something to bear in mind, all these changes should go out as majors/breaking so we can coordinate the release.