Go-ipfs: dag API and resolving paths

[vmx]

Version information:

go-ipfs version: 0.4.14-dev-25eeb1e25
Repo version: 6
System version: amd64/linux
Golang version: go1.9.2

Type: Enhancement, Bug

Description:

I'm currently working on implementing the dag API for https://github.com/ipfs/js-ipfs-api/. There I stumbled upon issues when running the https://github.com/ipfs/interface-ipfs-core/ dag tests. Resolving doesn't work as I expect it, though I might have a wrong understanding. Here's an example:

$ curl -F 'data={"data": "aGVsbG8gd29ybGQh"}' 'http://localhost:5001/api/v0/dag/put?hash=sha2-256&format=dag-pb&input-e
{"Cid":{"/":"QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz"}}
$ curl 'http://127.0.0.1:5001/api/v0/dag/get?arg=QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz'
{"data":"aGVsbG8gd29ybGQh","links":[]}
$ curl 'http://127.0.0.1:5001/api/v0/dag/resolve?arg=QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz/Data'
{"Message":"no link by that name","Code":0,"Type":"error"}

I would expect the last curl command to return:

"aGVsbG8gd29ybGQh"

Am I wrong?

[vmx]

This is based on a test interface-ipfs-core: in https://github.com/ipfs/interface-ipfs-core/blob/0c76cddcc7eaf8915ca11b20d6e41c7271b57bea/js/src/dag.js#L242-L247.

[vmx]

CBOR works as I'd expect it:

$ curl -F 'data={"hello":"world!"}' 'http://localhost:5001/api/v0/dag/put?hash=sha2-256&format=dag-cbor&input-enc=json'
{"Cid":{"/":"zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ"}}
$ curl 'http://127.0.0.1:5001/api/v0/dag/get?arg=zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ'
{"hello":"world!"}
$ curl 'http://127.0.0.1:5001/api/v0/dag/resolve?arg=zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ/hello'
{"Cid":{"/":"zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ"},"RemPath":"hello"}

[magik6k]

Yep, that's definitely a bug, it's something that should get resolved once https://github.com/ipfs/go-ipfs/issues/4666 is addressed.

[vmx]

I had a quick chat with @whyrusleeping on IRC about this. Here's the summary:

In the Go implementation slashes always mean "traverse links". So <cid>/somename means "follow the link with the name "somename". So the data of a ProtoBufs encoded node (here represented as JSON) would look like this:

{
  "Data": <some-base64-data>,
  "Links": [
    {"somename": { "/": <cid> }}
  ]
}

There is no way to get the data of a DAG node through a path.

[magik6k]

There is no way to get the data of a DAG node through a path.

Yes, but the other graph-related functions should still work. Git plugin is one example here.

In the Go implementation slashes always mean "traverse links"

This is the real problem here, it's going to be much simpler to address once /ipld is separated from /ipfs a. See #4666 for more details

[vmx]

@magik6k I've read #4666 before, but know I understand the /ipld - /ipfs separation. Does it mean that the test linked in my comment above https://github.com/ipfs/go-ipfs/issues/4728#issuecomment-367520360 should work once #4666 is solved?

[magik6k]

It will require some more changes to the internals. Dag api will use ipld resolving by default, so it will work.

[magik6k]

The todos here are:

• Solve #4666
• Rewrite dag commands to use CoreAPI
• Make merkledag nodes implement ipld properly and fix unixfs resolver

[magik6k]

We now have separate /ipfs and /ipld paths.

• Resolving /ipfs works as expected, as it's resolving unixfs paths and falling back to IPLD paths if resolver for a given node type is not available
• /ipld paths still need to be fixed (requires changes in go-merkledag)