Exoplayer: HLS seeking extremely slow

[NathanSass]

Issue description

I have an HLS playlist that has segments that are ~1000s. When seeking to offsets that are near the end of a segment (960s) seeking takes at least a minute. I would expect seeking to take only a few seconds. Especially when no decryption is required.

Reproduction steps

I have emailed the test app to dev.[email protected]
I have added a new section HLS SEEK ISSUES. Click on this section header and then choose Segment Size 1000 to see extremely slow seeking on startup.

Link to test content

emailed

A full bug report captured from the device

emailed

Version of ExoPlayer being used

2.10.1

Device(s) and version(s) of Android being used

Google Pixel, Android Version 9
It reproduces easily on any device.

[Romantic-LiXuefeng]

The player uses exact seeking when playing HLS stream. In details, Firstly the player needs to find the ts segment which the seek point owns to. Secondly, decode from the start of the ts segment, but drop the decoded frame until the seek point. So the seeking cost more time. Seeking to sync points is typically faster but less accurate than exact seeking. So you need to re_implement the HlsMediaPeriod. getAdjustedSeekPositionUs(long positionUs, SeekParameters seekParameters), return the position to which a seek will be performed. Like the MediaPlayer, it's just return the start position of the ts which the seek point owns to. The seeking operation is fast.

[ojw28]

Having ~1000s segments just sounds broken. That's ~166x what Apple recommend, which is ~6s. Segments are supposed to be relatively short.

[NathanSass]

@ojw28 I wasn't aware of that. We also tested with segments at 60s and found good seeking performance but downloading was very slow - which is why we moved segments to 1000s. I imagine that downloading with 6s segments would take absolutely forever. Is there some way to improve downloading speeds with smaller segments?

[ojw28]

We also tested with segments at 60s

60s is still way too long for segment duration. Small segments are required for HLS, because without them you lose all the benefits that HLS actually provides over regular progressive streams (e.g. the ability for the client to quickly adapt to changing network conditions during streaming playback). In which case you may as well just use a regular progressive stream.

I imagine that downloading with 6s segments would take absolutely forever.

There were some performance problems in ExoPlayer around downloading adaptive content. They were resolved in 2.10.0. If you're still using one of the @Deprecated SimpleCache constructors, you need to switch to one of the new ones to get the improved performance.

[NathanSass]

@ojw28 Our team decided to use 60s segments because our app is audio only & they deduced that 60s of audio only was equal to 6s of audio + video. Do you think this design decision would cause an issue with exoplayer?

I've tried downloading some content using exoplayer 2.10. I downloaded a 25min long audiofile that had 6s segments. It took almost 2 minutes to download. Our app deals with audiobooks that can be 40 hours long. I'm afraid that even using 2.10 could result in extremely long downloads using 6s chunks. For comparison, a 25min download with 60s chunks takes about 10seconds. Do you think HLS and exoplayer will be able to accommodate large downloads in a timely manner? Or would we be better off using another audio format?

[ojw28]

HLS doesn't really seem like the right tool for what you're trying to achieve, vs something much simpler like regular MP4. Is there a reason you're trying to use HLS?

[NathanSass]

The reasons for using HLS were:

• audiobooks are divided up into chapters & multiple audio files. The HLS playlist seemed to work well with this design.
• it seemed like a better choice for handling encryption & decryption
• It works with DRM solutions (We may want to implement in the future)
• We thought HLS would provide smoother transition from one audio file to another audio file. Something we have struggled with in the past. (Reasons are too long to enumerate)

Do you think we will be able to get the downloading performance from HLS that we are looking for? Will HLS downloading generally be a lot slower than downloading mp4s? A 20 hour audiobook should download in less than 5 minutes.

[ojw28]

I'll try and provide a high level summary of where we are with this, and hopefully in doing so answer your questions. I've not responded as clearly as I might have done so far!

Seeking

The response from @Romantic-LiXuefeng is basically the correct answer to this. Individual TS segments (i.e. the files that are listed in your HLS playlist) are not prefixed with any indexing information that lets a player efficiently seek to a position within them. So when the player wants to seek to a position it looks in the HLS playlist, finds the segment that contains the seek position, and then downloads and discards from the start of that segment up to the seek position, at which point it starts playing. This is normally fast, because HLS is designed with the expectation that segments are short in duration (e.g. ~6s is recommended). Seeking is slow in your case because you've deliberately made the segments much larger than is expected, which means the player may need to download and discard a large amount of data to reach a specified seek position. The speed of the seek will depend on whether the seek position happens to be near the start or end of the segment that contains it.

Downloading

Roughly the same amount of data will need to be downloaded in the HLS case as in the case where you have a single file (e.g. FMP4). This will be true regardless of the segment length. So there's no fundamental reason why one would be faster or slower than the other.

In practice, you're observing that shorter segment length significantly increases download time. This is likely due to round trip time when requesting each segment, and the fact ExoPlayer currently requests them one after another rather than attempting parallelization (or doing something else - see below). The data provided above for the 25 min long audio file would suggest you're incurring a per-segment overhead of around 0.5 seconds. That sounds pretty excessive. I tried measuring the time-to-first-byte to your server, and it was indeed about 0.5 seconds for me. Is there a reason it's so slow? I'm seeing closer to 0.1 seconds to a few random popular internet services that I tried. Just fixing that should make the situation significantly better.

In order to further improve HLS downloads when shorter segments are used, we could look at parallelizing segment downloads in ExoPlayer (e.g. having 5 segments being downloaded at once). This ensures the network remains fully utilized, which isn't the case currently. Another neat approach would be for you to arrange your media so that the segments are consecutive ranges of the same URL, like:

#EXTINF:10.00000,
#EXT-X-BYTERANGE:808400@0
audio-stream.ts
#EXTINF:10.00000,
#EXT-X-BYTERANGE:848068@808400
audio-stream.ts
#EXTINF:10.00000,
#EXT-X-BYTERANGE:811784@1656468
audio-stream.ts

This approach allows a smart enough client to merge arbitrarily many segment downloads into a single request. In theory a client could just make one request for audio-stream.ts and download the entire book, regardless of how short or long your segment durations are. ExoPlayer doesn't currently do this, but it's something we should look at. It's quite a lot more controlled than the parallelized approach. If you stick with HLS, I'd definitely suggest that you do this.

Format choice

I don't really understand the divided-into-multiple-files argument. Segmentation in HLS isn't (as far as I know) intended to be used to divide up content into logical pieces. It's intended to allow for adaptive switching, which isn't possible with your content anyway because there's only one quality provided anyway.

I'm assuming you control the media, so you could provide the entire book as a single large FMP4 file, which would be more efficient in nearly all ways (more efficient container vs TS, more efficient indexing vs HLS media playlists, avoiding 1x extra round trip at the start because you wont need a separate playlist file, etc).

Regarding DRM: FMP4 is used in DASH and can also be used in HLS, in both cases with DRM support. So I'd expect platforms to support DRM'd FMP4 streams directly. I'm not sure about that, however.

Conclusions

1. It would be nice if ExoPlayer could download long total duration, short segment duration content more quickly.
2. You should use the same URL and adjacent #EXT-X-BYTERANGE ranges to allow the client to merge requests for adjacent segments into a fewer number of larger requests.
3. I still have a feeling just using a single big FMP4 stream for the entire book would be significantly better. You'd have to think about this for yourselves though!

[ojw28]

For the first conclusion, I've filed https://github.com/google/ExoPlayer/issues/5978. I think that's the only actionable thing on the ExoPlayer side, so I'll leave this issue closed.