Ipfs-companion: Upload does not work with ipfs-cluster

[lidel]

How to Reproduce

Grab latest ipfs-companion and ipfs-cluster.
For the purpose of this bug it can just run locally in ephemeral docker:

$  docker run -it --rm --net=host ipfs/ipfs-cluster

Try to upload with API port in Companion pointing at:

1. go-ipfs (:5011) → OK
2. ipfs-cluster (:9095) → SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 2 column 1 of the JSON data

The Actual Problem

TL;DR is that companion uses js-ipfs-api (HTTP API Client library), and the library decides how to parse response payload based on additional HTTP headers.

Cluster does not seem to return X-Chunked-Output and that is why chunkedObjects in this line is false, which changes the flow and triggers parser error in streamToJsonValue.

How to Fix?

@hsanjuan IIUC a fix would be to add X-Chunked-Output (and X-Stream-Error?) to ipfs-cluster's response to match what go-ipfs does.

See comparison below.

| go-ipfs | ipfs-cluster |
| ---- | ---- |
| | |

[hsanjuan]

Will do @lidel ! Thanks

[lidel]

Re-tested and can confirm this was fixed works with Firefox and ipfs-cluster v0.7.0 :tada:

[hsanjuan]

@lidel actually it's still broken with chrome because header gets stripped because CORs is not set correctly by cluster. :/

[lidel]

@hsanjuan hm.. this surfaces an interesting discrepancy between browser vendors.

I tested scenario where:

• go-ipfs v0.4.18 is at 127.0.0.1:5001
• ipfs-cluster v0.7.0 is at 127.0.0.1:9095
  (run without NAT via docker run -it --rm --net=host ipfs/ipfs-cluster:v0.7.0)
• ipfs-companion v2.5.1.12010 has API port set to 127.0.0.1:9095

And.. quick upload in Firefox works fine even without CORS headers:

| :9095 in Firefox :angel: | :9095 in Chrome :anger: | :5001 in Chrome :angel: |
| ---- | ---- | ---- |
| | | |

[hsanjuan]

The inspector shows the X-Chunked-Output headers everywhere, but it Chrome Javascript doesn't see it unless the Access-Control-Expose-Headers is set accordingly. We'll have a better approach to headers in the proxy endpoint for the next release and fix this once and for all.

[hsanjuan]

@lidel as of cluster 0.8.0-rc1 this is finally really fixed. Can be closed.

[lidel]

Confirmed with docker run -it --rm --net=host ipfs/ipfs-cluster:v0.8.0-rc1, works as expected :+1: