Dash.js: Cache detection logic returns a lot of false positives

[AxelDelmas]

This method is supposed to detect when a segment comes back from cache.
I see that the latency threshold has already been removed since I last checked, which is good since it was the main culprit IMO.
But I also noticed a lot of false positives using the download time thresholds. Also, it does not account for the bitrate so it seems highly probable for me that fragment download from low bitrates will always be detected as coming from cache.

Isn't there a more reliable way to figure out if a request indeed comes from cache? I've found some stuff regarding looking up the Date header but it seems some CORS response header configuration is needed to make that header accessible from JS.
It also seem that it's possible to use the Resource Timing API to figure out if a resource comes from cache, using transferSize and encodedBodySize which do not seem to have cross-domain restriction (but I'm not sure exeactly which browsers support these properties).

[davemevans]

FYI we have found resource timing doesn't work very well for range requested resources - see https://github.com/w3c/resource-timing/issues/102.

[AxelDelmas]

We've also been able to observe the following scenario:

1. download segments at the highest quality. download time for video is above the 50ms threshold
2. throttle the bandwidth to simulate a bandwidth drop. The quality goes down to the lowest bitrate. The download time is still above 50ms due to the low bandwidth available
3. Stop throttling the bandwidth after some time. The player is still playing the lowest quality, but this time has a high bandwidth so the download time is now <50ms.
   All new bandwidth measurements are discarded due to the cache detection mechanism, so the historical bitrate value is never updated and stays very low. As a consequence, the player is stuck on the lowest bitrate.

This scenario can happen for real users when they experience a momentary bandwidth drop.

I'd say the current implementation is broken, at least with the current thresholds. I think we should at least make the thresholds configurable by the developper (I've got a PR ready for that). And I think the defaults should be much lower than that. If a segment indeed comes from cache, the download will be virtually instantaneous, so I think a threshold of 2 or 3ms should enable cache detection while having much less false positives.

It would be even better to use a more reliable detection mechanism if feasible (thanks @bbcrddave for the heads up regarding this ressource timing issue).

[AxelDelmas]

Any update on this? At the very least I can make a PR to make the thresholds configurable on the player API, so users are able to test different configurations and see what works best for them.
I'd also suggest to lower the default thresholds to 2 or 3ms, to avoid pathological edge cases due to false positive like the one I brought up above.

(Adding link to related solved issue https://github.com/Dash-Industry-Forum/dash.js/issues/1931)

[epiclabsDASH]

Hi @AxelDelmas. Very interesting discussion. IMHO, I would prefer improving the mechanism for detecting cached responses than letting developers choose the threshold (IMHO these values should be transparent for dash.js users).

I agree with you there is space for improvement here. Without the need of using new browser apis that could be incompatible with some browsers or type of requests, there are things we can do to improve the "heuristic" for detecting cache requests while keeping it simple.

I need to do some testing but a combination of what we are doing today, plus taking into account chunkSize, plus improving the defined thresholds could help to reduce the number of false positives.

Any opinion will be welcome.

[epiclabsDASH]

@AxelDelmas, after doing some empirical tests I couldn't find default values that works better than the current ones. At least not in a generic way. Lowering values I could reduce the number of false positives in some computers but I also increases the number of false negatives in some others (mainly in lower end devices).

So, instead of modifying the default values I just added a new method to the API so users can adapt thresholds to their needs (setCacheLoadThresholdForType).

Thanks

[AxelDelmas]

Ok seems good to me. Thanks!