Brave-browser: WebTorrent should not store torrent data in memory

[feross]

Description

WebTorrent seems to be keeping the entire torrent in memory as the file is downloaded. I can see the RAM usage of the "Brave Browser Helper" process increase along with the amount of the torrent that is downloaded, which should not happen. It appears that it is being initialized with the in-memory storage mode instead of the filesystem mode. This is because webpack is used to build the JS for the WebTorrent extension and that causes WebTorrent to substitute certain packages using the package.json "browser" field.

May be related to: https://github.com/brave/brave-browser/issues/5372

Steps to Reproduce

1. Visit http://mirror.math.princeton.edu/pub/mageia/iso/7.1/torrents/Mageia-7.1-x86_64.torrent
2. Start the torrent download
3. Open Activity Monitor.app on Mac and observe memory usage climbing in proportion to the amount of torrent that is downloaded.

Actual result:

Torrent is stored in memory instead of on the disk.

Expected result:

Torrent should not consume RAM in proportion to the torrent size, since this means that users with e.g. 8GB RAM are going to have a sad time when they download an 8GB+ torrent.

Reproduces how often:

Easily reproduced

Brave version (brave://version info)

All Chromium versions.

[btlechowski]

Similar/Dupes issues https://github.com/brave/brave-browser/issues/1199 https://github.com/brave/brave-browser/issues/1228

[sheddup]

Is this being looked at? Can something like https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API be used instead of RAM

[FTOH]

Is this being looked at? Can something like https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API be used instead of RAM

I would prefer it should store files in Downloads folder to be able open with other players (for media).

[PopTudor]

I looked into the same thing these days. I believe the answer you are looking for is to use an Abstract Chunk Store implementation. Unfortunately idb-chunk-store has some issues when I tried to use it last day. The one that worked for me(but not perfect) is this one. The thing is that both of these were not updated since last 3+ years ago so I don't know if it could be improved. My usecase is to seed large files(5GB for example). Using memory-chunk-store it crushes my RAM and occupying 8GB in memory data. With indexeddb-chunk-store goes to like 5GB but it also takes a bit more time to finish the seed(since it writes to disk). For browser I'm not sure anything can be done because of lack to user FS.
I also tried the webtorrent-cli (which has access to the FS) and is almost instant when seeding a file with no memory footprint(that I could see in activity monitor) so my guess is that is using the FS-chunk-store. I'm quite new to this project(and browser/javascript tech in general) & some confirmation from someone in the official team would be great. I know the thread is for Download but I think is the same issue for seeding

[PopTudor]

update: idb-chunk-store seems to be fine after removing package-lock.json and npm install again. However the seeding of a 5GB file still takes 5GB of RAM

[MatheusPese]

This issue is bad because my pc just freezes completelly when downloading torrents with brave because of it. My RAM usage goes to 100%

[srirambv]

This should probably be prioritized and fixed soon. Torrent feature becomes useless if the browser starts to freeze up and download is lost.

cc: @rebron @bsclifton @feross

[AerosHawk]

Can confirm this is still an ongoing issue.
I was also trying to find out if using webtorrent inside Brave protects you from sneaky ISPs, the same way that a VPN does..? Sorry, side point, but wasn't really able to find an answer and it led me here.