Exoplayer: Improve speed of DASH/HLS/SS downloads

[workspace]

I have implemented Offline Playback using DownloadService of 2.8.0, but there are some things that should be improved. When user downloads a 50-minute video using Dash, thousands of chunks are downloaded. Sometimes it takes about 30 minutes(700MB) on gigabyte wifi(But it has many reasons to explain. e.g media service status). Currently DownloadService supports downloading several videos at the same time. How about applying simultaneous downloads to Chunk? When downloading chunks, downloading chunks simultaneously through multithreading will improve download speed of single video.

This sequential downloading must be optimized.

[erdemguven]

Thanks for your suggestion, we'll look into this. In the meantime, you can create separate DownloadActions for different tracks and/or different periods you want to download, to make them downloaded in parallel.

[workspace]

I already allow user 4 simultaneous downloads. Create separate downloadActions for each track looks nice solution, however, it is hard to show correct download progress of whole video, or to validate if it is playable... anyway, thank you for your response :)

[tmizuguchi-bckk]

@erdemguven
Any update on this? It is very important feature for us to improve download offline playback on our service.

[ofuwape]

@erdemguven @ojw28 Any update on this feature?

[cody-ted]

@erdemguven wondering if you have this in the schedule? Our biggest current user complaint is the slow downloading speed of TED Talks (I work on the TED Android app). The performance isn't the network, but the overhead in handling all the HLS file segments. We might consider building ourselves, but would be a bit tricky - would love an update if you have one. Thanks!

[ofuwape]

@erdemguven @ojw28 Any update on this feature?

[ojw28]

Before we do anything, we (or someone) should be measuring why downloads are actually slow. I'm not completely convinced parallel segment downloads is the single biggest thing that could be done to improve thing (vs, for example, optimizing CacheContentIndex). Hence I've renamed this issue to specify just the problem, not also the solution.

[janeriksamsonsen]

I can confirm that download is extremely slow, even when using a separate downloadtask for each track (video, audio and subtitle). The behavior can be observed using OkHttp and Stetho. There seems to be some kind of parallelisation, but the delay between finishing one segment and downloading the next is becoming larger for each segment downloaded. In our tests the movie is ~90 minutes with 2 second segments, causing the total number of downloads for all tracks (video, audio, subtitle) to be ~8100. The CPU usage of the device is also very high ( >50%) at the end. Android Studio profiler indicates that most of CPU processing happens in com.google.android.exoplayer2.upstream.cache.CachedContentIndex.writeFile().

All numbers from Nvidia Shield TV with 300mbit connection.

So before attempting to download multiple chunks/segments in parallel i would recommend to investigate what causes all the waiting and CPU usage.

[erdemguven]

We're currently working on using SQLite for cache index which will greatly improve index updates. It'll be ready soon. We'll update this thread when it's available.

[kartdroid]

We're currently working on using SQLite for cache index which will greatly improve index updates. It'll be ready soon. We'll update this thread when it's available.

Is there any PR available for this that we can have a glance at ?

[ojw28]

SQLite based cache indexing is enabled in dev-v2, including in the demo app. Please give it a try. If you want to try it out in your own app then you'll need to migrate to one of the new SimpleCache constructors that takes a DatabaseProvider.

I'm going to close this as fixed, but if performance is still poor then please do let us know.

[janeriksamsonsen]

Download speed is much faster now (and CPU load is back to normal). Great work!

Since this is in place, the next logical step would probably be to allow parallelisation of tracks within a download to improve speed even further.

[ojw28]

I filed https://github.com/google/ExoPlayer/issues/5978 for exploring parallelization, and also merging segments into fewer HTTP requests when they have the same URL and adjacent byte ranges.

[Stijak]

Download speed is much faster now (and CPU load is back to normal). Great work!

Since this is in place, the next logical step would probably be to allow parallelisation of tracks within a download to improve speed even further.

Hi @janeriksamsonsen
Did you do anything special in your implementation. We did clean implementation like in their Demo app, and exoplayer 2.10 only improved download speed for about 10%. 700MB video (8000 chunks) still takes 40 minutes over fast WI FI.
Also, what HTTP are you using. Our content provider is using HTTP/1.1 so with handshake improvements maybe HTTP/2 could be a little faster.
Hope #5978 will improve it, otherwise we will have to implement download from the scratch, since it is like 10 times slower then our legacy implementation.

[janeriksamsonsen]

@Stijak - We did not really do anything special to get the performance gain. The download speed has not changed for individual segments, but because the index writing in SimpleCache is so much better the overhead (CPU/IO-load between segments) is much lower, and hence the total download time is reduced multiple times. Are you by any chance downloading to SD-card (that will be a bit slower i guess)? We're using OkHttpDataSourceFactory when creating DownloadManager, but don't know if that makes a difference.

[akavrt]

I've tested download speed with v2.10 thoroughly and can confirm significant increase on devices with slow storage. Meizu M5 Note is one of our test devices where download speed with ExoPlayer 2.9 is painfully slow — it takes more than 40 minutes to load our 1.5 Gb test DASH asset on fast Wi-Fi while Samsung S8 loads same test asset in 4 minutes. With ExoPlayer 2.10 both devices load that asset in comparable time, around 4 minutes on average.