Js-ipfs: benchmark.js.ipfs.io

[goga-m]

First of all, great work with js-ipfs!

I have been trying to implement file streaming with ipfs in browser and face a performance lag when trying to stream large files with js-ipfs in the browser.

To simplify the case and not to add any custom code implementations, I am following the example of transfering files between browser and other ipfs nodes here:
https://github.com/ipfs/js-ipfs/tree/master/examples/exchange-files-in-browser

So this example works nicely when working with small files, but when I try to fetch larger ones, I get a performance lag.

Scenario 1 (Websockets connection):

• I start a go-ipfs daemon (let's say node1) with websockets connection enabled
• I add a file sized ~4MB locally with the CLI.
• I connect a browser-node (node2) with the go-ipfs node and request the file.

The file stream is finished after approx. 36-39 seconds, and for some reason the browser-node drains the cpu resources, and sometimes craches the browser.

Scenario 2 (WebRTC connection):

• With the two nodes open, I open a third browser-node (node3) (incognito window) and
• connect it with the already opened browser-node (node2) using WebRTC.

The file stream is also finished after approx. 36-39 seconds, and the cpu is also drained.

Scenario 3 (WebRTC & Websockets connection):

• With the two nodes open, I connect the third browser-node (node3) with both node1 and node2 that already have the file.
• I request the file and have the same results: the file stream lasts ~40 seconds and the cpu resources issue still remains.

I have the latest update of js-ipfs (latest commit: a85cf70), and I tried the scenarios again with the same results.

Any ideas or directions of what causes this issue, and how to overcome this?

Many thanks!

[ya7ya]

Yeah, I'm seeing the same issue. :+1:

[dignifiedquire]

I have been doing some perf improvements around secio, which seems to be the largest offender in terms of raw transfer speed in the browser right now here: #128

[daviddias]

Thanks for the great report, @goga-m. @moshisushi and @dignifiedquire are now investing time into getting some improvements in our crypto channel with some significant results already.

It would be extremely helpful if you can convert your tests into runnable benchmarks we can all use. @pgte did a lot of foundational work to get a hardness for perf testing here: https://github.com/ipfs/ipfs-performance-profiling, however, it currently focus on local operations and not as much in network (where secio is). We need to change that :)

[goga-m]

Thank you for your quick answers @diasdavid and @dignifiedquire and for targeting the cause of this. @ya7ya thanks for testing too. I am looking over your references and looking forward to find some time to prepare network-performance runnable benchmarks.

All the best

[goga-m]

Greetings guys,

I started creating a simple utility to check ipfs's stream performance for the GET method and see some numbers and metrics.

Here is the repo
https://github.com/goga-m/js-ipfs-browser-performance
and the online demo:
https://goga-m.github.io/js-ipfs-browser-performance/

I hope it will be useful.
Let me know if for any ideas, additions, improvements or bugs.

I updated to the newest version 0.26.0, I compared it with the previous one (0.25.4) and saw a much better performance, especially in the browser crash issue.

All the best

[victorb]

@goga-m that's a really cool example! Looks good and works well too 🎉

Would be cool to see the option of running one test many times and taking the average/median from many test runs. Also the ability of clearing the cache (un-pinning, if we had it) between the runs, so we can test both fetching externally and locally.

Otherwise, great utility, many thanks for writing it and sharing it with us! 🍨

[goga-m]

Thanks guys and @VictorBjelkholm thanks for your good words :-) Glad to help!

Now that the browser crash issues is resolved, it makes sense to create multiple test runs. I will look forward to add it at the next update along with the unpin option.

Thanks for the ideas

Cheers!

[daviddias]

@goga-m just realized I had missed your example all this time! Shame on me, it looks RAD!

Can we make your repo part of this repo into a benchmarks folder? Would love to have that always around for more testing. Also, I think you will like https://github.com/ipfs/js-ipfs/pull/1086#issuecomment-345365729

Ideally, we could host this at benchmark.js.ipfs.io and make it a growing set of benchmarks :)

We can also make it its own repo in the ipfs org :)

[goga-m]

That sounds fantastic @diasdavid!

Really happy to see this https://github.com/ipfs/js-ipfs/pull/1086#issuecomment-345365729 !

Yes It could be easily grow to support other features as well, and I would love to help on this.
So do you want to move this repo on ipfs.org, and start building there?

All the best!

[daviddias]

@goga-m added you to the Performance and Benchmarking team https://github.com/orgs/ipfs/teams/performance-and-benchmarking/discussions, once you join you should be able to move the repo. Then we can set up the subdomain and start deploying and evolving the benchmarks :)

[goga-m]

Perfect @diasdavid, so let's move the conversation there.
Thanks!

[daviddias]

Yes, let's do it. The first step is to make it publishable https://github.com/ipfs/benchmark.js.ipfs.io/issues/2