Webtorrent: ERR_CACHE_OPERATION_NOT_SUPPORTED Error on Chrome

[mikeal]

What version of WebTorrent?

0.98.19

What browser and version? (if using WebTorrent in the browser)

Chrome 62 on Mac

Reproduce

<script src="https://cdn.jsdelivr.net/npm/webtorrent-component@latest/dist/webtorrent-component.min.js"></script>
<web-torrent src="magnet:?xt=urn:btih:08ada5a7a6183aae1e09d831df6748d566095a10&dn=Sintel&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2F&xs=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2Fsintel.torrent"
    file="Sintel.mp4"
    />

This is a component I just wrote for loading webtorrent content. It's pretty simple, but in the process of testing and loading this creative commons video a TON I noticed that every 10th time I load the page Chrome gives me an error and the video eventually stops.

VM1140:1 GET https://webtorrent.io/torrents/Sintel/Sintel.mp4 
net::ERR_CACHE_OPERATION_NOT_SUPPORTED

[DiegoRBaquero]

I tested and received one error only, on a range of bytes that hadn't been requested before. range:bytes=123188-254259 Reloading the page would sometimes lead to no-error loading.

This is gonna be a tough one to debug. Good news tho, forcing non-caching with a query seed seems to work: https://github.com/igvteam/igv.js/issues/424#issuecomment-336336788, maybe using cache-control headers should work too

[jrobinso]

Hi @DiegoRBaquero I implemented the igv query seed solution, however it's not working with Amazon sigV4 signed URLs because the query differs from the url that was signed. Did you try cache-control headers? Or come up with any other solution (other than a random query seed)? Its disappointing that this has not been fixed in Chrome.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[feross]

Spent a little while debugging this with @jhiesey.

We were able to reproduce it in Chrome 66. There's not really anything we can do about ERR_CACHE_OPERATION_NOT_SUPPORTED I think. We're pretty sure this is not a webtorrent or stream-http bug. It's a Chrome bug.

It seems that Chrome's cache is too clever for it's own good. When we make an HTTP range request and Chrome already has part of the requested range in it's cache, it tries to produce a response synthesized from (1) the data from it's cache and (2) data from a new HTTP request that it makes. Presumably this is done to avoid fetching more data than is needed.

We confirmed this was happening because when you enable the "Disable cache" setting in DevTools, we see that ALL requests are for the full piece length (131072 bytes). But when you allow the cache to be used, lots of the requests are for much shorter ranges of data. These are ranges that we did not request from WebTorrent.

It seems that their cache code fails to actually recombine these data into a valid response once in a while and when that happens we get an error from the fetch() API.

---

I would say that the trail ends here and we could close this issue, but the plot thickens...

As @jhiesey and I dug into this, we learned that WebTorrent fails to re-fetch pieces from peers when the fetch() fails and the request object emits an 'error' event like it does with this ERR_CACHE_OPERATION_NOT_SUPPORTED error.

This is where the http request is made: https://github.com/webtorrent/webtorrent/blob/master/lib/webconn.js#L57

When a failure happens, this is the line that runs: https://github.com/webtorrent/bittorrent-protocol/blob/6b3819cf671a352bb12c05491cf8556608d8d21e/index.js#L501

That prints a debug message and returns early.

So, I'm leaving this issue open so we can investigate why the prioritization code does not re-request this piece from the peer. Instead it just remains missing and the torrent never completes.