Go-ipfs: ipfs object diff Error: expected protobuf dag node

[ivan386]

Version information:

go-ipfs version: 0.4.13-
Repo version: 6
System version: amd64/windows
Golang version: go1.9.2

Type: Bug

Description:

>ipfs object diff -LD QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975 QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL
2018-02-12 19:02:21.703125 INFO cmd/ipfs main.go:301: IPFS_PATH .ipfs
2018-02-12 19:02:21.703125 DEBUG cmd/ipfs main.go:295: calling pre-command hooks...
2018-02-12 19:02:21.703125 DEBUG cmd/ipfs main.go:342: executing command locally
2018-02-12 19:02:21.734375 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmS7RSorsfu1MJpBjcu7L1W2QfsoFDKSrvqAiHzLY4EUH4'
2018-02-12 19:02:21.734375 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmWWC1GHCa7at4K6oMK7Wm7odYCphhRvXYXKAJBreMhRLu'
2018-02-12 19:02:21.734375 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmSKboVigcD3AY4kLsob117KJcMHvMUu6vNFqk1PQzYUpp'
2018-02-12 19:02:21.750000 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn'
2018-02-12 19:02:21.750000 DEBUG path resolver.go:54: Resolve: '/ipfs/QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975'
2018-02-12 19:02:21.750000 DEBUG path resolver.go:139: resolve dag get
2018-02-12 19:02:21.750000 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975'
2018-02-12 19:02:21.750000 DEBUG path resolver.go:54: Resolve: '/ipfs/QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL'
2018-02-12 19:02:21.750000 DEBUG path resolver.go:139: resolve dag get
2018-02-12 19:02:21.750000 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL'
2018-02-12 19:02:21.765625 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'zb2rhmy65F3REf8SZp7De11gxtECBGgUKaLdiDj7MCGCHxbDW'
2018-02-12 19:02:21.765625 DEBUG blockservice blockservice.go:160: BlockService GetBlock: 'zb2rhoLDGJwhWakT8nTZKgazBJKY7aWvcn6csxb7FCJwfRFK3'
Error: expected protobuf dag node
2018-02-12 19:02:21.765625 INFO cmd/ipfs main.go:240: Shutting down node...
2018-02-12 19:02:21.765625 DEBUG core core.go:531: core is shutting down...
2018-02-12 19:02:21.765625 DEBUG blockservice blockservice.go:251: blockservice is shutting down...

Additional information:

>ipfs dag get QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975
{"data":"CAIS4AMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAYwBYg4AMg4AMg4AMg4AMg4AM=","links":[
{"Name":"1","Size":0,"Cid":{"/":"zb2rhmy65F3REf8SZp7De11gxtECBGgUKaLdiDj7MCGCHxbDW"}},
{"Name":"2","Size":480,"Cid":{"/":"zb2rhb3bbxnrqKS2QopcDvbCz5F9nRW7MFzRkhH7LoqhG8AyD"}},
{"Name":"3","Size":480,"Cid":{"/":"zb2rhahcgW9tVL7cbPnbdf8y5DzYdbLqCKh2AeBto1JnyCErj"}},
{"Name":"4","Size":480,"Cid":{"/":"zb2rhbo1XsFWFP3LCF14NHDikCBn1o8hXVZ6BtR5hvntU5PRp"}},
{"Name":"5","Size":480,"Cid":{"/":"zb2rhhkerjTtMxmCwrT9UZC3z7vVuLK8nPnMJzBFRwrEQtiyL"}}
]}

ipfs dag get QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL
{"data":"CAIS4AMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAY6BY=",
"links":[
{"Name":"1","Size":480,"Cid":{"/":"zb2rhoLDGJwhWakT8nTZKgazBJKY7aWvcn6csxb7FCJwfRFK3"}},
{"Name":"2","Size":480,"Cid":{"/":"zb2rhb3bbxnrqKS2QopcDvbCz5F9nRW7MFzRkhH7LoqhG8AyD"}},
{"Name":"3","Size":480,"Cid":{"/":"zb2rhahcgW9tVL7cbPnbdf8y5DzYdbLqCKh2AeBto1JnyCErj"}},
{"Name":"4","Size":480,"Cid":{"/":"zb2rhbo1XsFWFP3LCF14NHDikCBn1o8hXVZ6BtR5hvntU5PRp"}},
{"Name":"5","Size":480,"Cid":{"/":"zb2rhhkerjTtMxmCwrT9UZC3z7vVuLK8nPnMJzBFRwrEQtiyL"}}
]}

[leerspace]

Sorry if it's implied in the first post, but I think this happens when trying to diff objects that were created with --raw-leaves. I'm guessing ipfs object diff doesn't have support for raw leaves.

[schomatis]

Yes, exactly:

https://github.com/ipfs/go-ipfs/blob/9a21a8cb1d2f1cda815e97502ba231731ed0785d/dagutils/diff.go#L135-L143

[overbool]

Hey @Stebalien , @schomatis , if anyone else fix this issue? if not, can I do for it?

[schomatis]

@overbool Yes please! Ping me if you have any questions.

[Stebalien]

Note: The ipfs object commands are designed to work on ProtoNode (protobuf) dag nodes only. We're eventually planning on killing off these commands (and adding a more general purpose ipfs dag diff command that operates on arbitrary IPLD data) but adding support for Raw nodes to this command sounds like a reasonable thing to do for now.

[overbool]

@schomatis I see the problem after you explained, but i don't know why it will print Error: expected protobuf dag node. Because it doesn't handle the err in object/diff.go

https://github.com/ipfs/go-ipfs/blob/e4e35047bd338029fd2c4ff25c71bc6cccc35bf3/core/commands/object/diff.go#L76-L78

[schomatis]

Thanks @overbool you're right, we're not handling that error and we're silently failing, I'll submit another issue about it, feel free to take it.

[schomatis]

Submitted https://github.com/ipfs/go-ipfs/issues/5468 to fix the ignored error, and while Diff in fact does not support raw leaves I'm not sure that that is the problem of this issue, @ivan386 could you provide the content of

• QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975
• QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL

(which I'm not being able to find) to try to reproduce this.

[overbool]

@schomatis I also can't reproduce this issue

[ivan386]

@overbool @schomatis I upload blocks to ipfs.io.
https://ipfs.io/ipfs/QmRqh1ckCLG6nXoE1aJQT2CHrByk5DWji8v3s4bHcmN975
https://ipfs.io/ipfs/QmUeec6DpWz2CA9uZ7GgPAB5nc6QYDg8yax4Wapxs1LSQL

[schomatis]

Thanks @ivan386 , I can now test this. It seems that the error check existed before the CoreAPI was used in this command but was lost in the changes, it has now been incorporated back again in https://github.com/ipfs/go-ipfs/pull/5469. The problem is indeed in the lack of support for raw leaves in the Diff function which will be tracked in https://github.com/ipfs/go-ipfs/issues/5470, so I'm closing this issue.

[schomatis]

@Stebalien Do you prefer to keep this one open and close #5470?

[Stebalien]

Either way, I just didn't notice that one so I reopened this one and then closed it when I noticed.