Js-ipfs: Add support for --only-hash - Calculating a multihash without adding the content to IPFS

[fazo96]

• Version: 0.27.7
• Platform: any

Type:

Enhancement

Severity:

Low

Description:

go-ipfs has the --only-hash option when calling ipfs add. It is useful in the strange case where you need to store a multihash of some data somewhere before actually adding the data to IPFS.

Maybe this is already possibile? I tried looking around but it seems the option is only in go-ipfs. I thought about adding the data then deleting from the local node but it looks like this can't be done either

[mitra42]

While I don't need this currently, this was something one of our earlier demos wanted. The scenario was building a data structure that referred to a IPFS, I wanted something that would synchronously return the hash to add to that data structure while I was in the background asynchronously adding the content.

[fazo96]

We found that using https://github.com/ipld/js-ipld-dag-pb (for protobuf) and https://github.com/ipld/js-ipld-dag-cbor (for cbor) directly allows us to easily compute hashes, however this cannot be used when adding a directory tree, only with small self-contained pieces of information

[JonKrone]

@fazo96, there was some discussion of --only-hash in https://github.com/ipfs/js-ipfs-api/issues/509. Is that what you're looking for? I looked at adding it and it's on my list but haven't found the time yet to work on it. You're welcome to take it on if you're interested :)

[fazo96]

@JonKrone yes, that's exactly what I was looking for, I think it should also be implemented in js-ipfs directly not only in the api client

For now my problem is solved by creating DAGNodes directly, I'd still like to contribute but I need to see if I can find the time

[alexanderattar]

@fazo96 can you elaborate a little using DAGNode.create to compute the hash? I so far have been unable to get hashes to match from node.files.add. Here's an example that demonstrated different CIDs for each output:

'use strict'

const series = require('async/series')
const IPFS = require('ipfs')
const {
  DAGNode,
  DAGLink
} = require('ipld-dag-pb')

const node = new IPFS()
let fileMultihash

series([
  (cb) => node.on('ready', cb),
  (cb) => node.version((err, version) => {
    if (err) { return cb(err) }
    console.log('Version:', version.version)
    cb()
  }),
  (cb) => DAGNode.create(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash:', mh)
    cb(null, mh)
  }),
  (cb) => node.files.add(Buffer.from('Hello'), (err, filesAdded) => {
    if (err) { return cb(err) }
    console.log('\nAdded file:', filesAdded[0].path, filesAdded[0].hash)
    fileMultihash = filesAdded[0].hash
    cb()
  }),
  (cb) => node.files.cat(fileMultihash, (err, data) => {
    if (err) { return cb(err) }
    console.log('\nFile content:')
    process.stdout.write(data)
  })
])

[fazo96]

@alexanderattar In my case what I needed was the same result between creating a DAGNode (without storing it in IPFS) using DAGNode.create(buffer) and the result of calling ipfs.object.put(buffer) where the buffer is identical between the two.

This works, I modified your example to show it:

'use strict'

const series = require('async/series')
const IPFS = require('ipfs')
const {
  DAGNode,
  DAGLink
} = require('ipld-dag-pb')

const node = new IPFS()
let fileMultihash

series([
  (cb) => node.on('ready', cb),
  (cb) => node.version((err, version) => {
    if (err) { return cb(err) }
    console.log('Version:', version.version)
    cb()
  }),
  (cb) => DAGNode.create(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash (dagnode):', mh)
    cb(null, mh)
  }),
  (cb) => node.object.put(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash (put):', mh)
    console.log('\nBuffer (put):', dagNode.toJSON().data)
    cb(null, mh)
  }),
  (cb) => node.files.add(Buffer.from('Hello'), (err, filesAdded) => {
    if (err) { return cb(err) }
    console.log('\nAdded file:', filesAdded[0].path, filesAdded[0].hash)
    fileMultihash = filesAdded[0].hash
    cb()
  }),
  (cb) => node.object.get(fileMultihash, (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    console.log('\nBuffer (get after add):', dagNode.toJSON().data)
    cb(null, dagNode.toJSON().multihash)
  }),
  (cb) => node.files.cat(fileMultihash, (err, data) => {
    if (err) { return cb(err) }
    console.log('\nFile content:')
    process.stdout.write(data)
  })
])

I also tried to diff the data of the content added using files.add and added using object.put in the same script. It looks like there are some bytes added before the actual content when files.add is used which would explain the different hash.

I am not sure why this happens though, probably the files API is higher level and does some extra work

[alexanderattar]

Awesome! Thank you @fazo96. This is helpful

[JonKrone]

Merged to master, will be available at next release :).

[ANUDAVIS]

I have a doubt.. Can anyone please help me to clear..
I am asking a case: We can hash a file or data using multihash or sha256 and whether we can retrieve the same data or file from the hash obtained?
or any case or methods are there inorder to retrieve the file or data from hash without using IPFS?
or is there any encryption method which encrypts a 5mb file and outputs some hash like 32 bytes content so that we can retrieve the file from that 32 byte content?

[haywirez]

Is there a simple, lightweight and reliable way to compute a file's IPFS hash right now? While I'm not using IPFS at the moment, I'd still like to add an info header with the IPFS hash to all uploads in my storage backend.

[alanshaw]

Does this help @haywirez? https://www.npmjs.com/package/ipfs-only-hash

[xmaysonnave]

I'm looking for a --only-hash while adding a dag-pb node through IPLD.
The ipfs.add or ipfs.addAll have such optional onlyHash attribute.
However ipfs.dag.put doesn't seem to have it.
Am I missing something ?
Thanks