Video.js: SSAI live stream playback freezes on Tizen TV after ~30s due to bug in native streaming engine

[kevinscroggins-youi]

Description

We have been observing an ongoing issue for several months involving Google HLS live streams with server side ad insertion causing Tizen's native video player (AVPlay) to freeze in web-based TV applications after approximately 30 seconds. After significant research digging through the device logs, the issue appears to be related to the underlying streaming engine and curl implementation where re-directed URLs can trigger an immediate timeout and cause the player to end up in a bad state where playback freezes entirely and you have to re-start the whole player which is far from ideal. Samsung has unfortunately not been very helpful in finding a resolution to the problem, so we have sought after integrating various alternative MSE/EME HTML5/JavaScript based players such as Video.js in an attempt to circumvent the issue. Thus far we have had the most success with Video.js both actually playing the stream and actually not freezing on very specific firmware on at least one Tizen TV. We also observed that in the native device logs, Video.js appears to be able to detect and leverage the built-in native HLS support and it uses that, which leads to us observing the same failure and ultimate ~30s live stream freeze even on Video.js. There also appear to be some configuration options that allow us to override the native HLS implementation technology to seemingly use a pure JavaScript solution instead. We have included configuration parameters to try and override the native implementation to circumvent this issue, but based on our logs the code that overrides this in the HlsHandler does not appear to be executing and we are still observing the freeze. Any assistance or additional information in avoiding usage of the native HLS implementation would be much appreciated. It is also worth noting that the official Google HLS SSAI test stream works fine and does not freeze, but there are several other SSAI live streams that do freeze so it is a fairly widespread issue. Thank you!

Player configuration:

            self.player = videojs(self.container.id, {
                controls: false,
                autoplay: false,
                loadingSpinner: false,
                html5: {
                    overrideNative: true
                }
            });

Steps to reproduce

Explain in detail the exact steps necessary to reproduce the issue.

1. Send a POST request to obtain the stream manifest from the following URL: https://dai.google.com/linear/hls/event/ixE_JVvSSCWyxU7eXJoV-A/streams
2. Load SSAI encoded live stream, ideally one with Google DAI
3. Load manifest into Video.js and play within Tizen TV web application
4. Observe that stream usually freezes after approximately 30 seconds of playback

Results

Expected

That the live stream plays successfully with no freezing for an extended period of time.

Actual

The live stream freezes.

Error output

There is no informative output from Video.js, but we did obtain native device logs when observing the same stream freeze using Tizen's AVPlay API (native player) which uses the same underlying streaming engine:

E/STREAMING_ENGINE(  916): Curl.cpp: ResultIf(1974) > [Warning] CURLE_OPERATION_TIMEDOUT occured!
E/STREAMING_ENGINE(  916): Curl.cpp: ResultIf(1989) > Curl Download Error!!: pause(0), interrupt(0), has(-53), Curl(28), http(200), now(2168732), total(3840276), https://rcavlive-dai.akamaized.net/hls/live/696614/cancbftprem/20190819T033438/master_5000/00010/master_5000_00935.ts, curlID(-1)
E/STREAMING_ENGINE(  916): Curl.cpp: ResultIf(2014) > Curl Retry on operation timeout: retry(0), pause(0), interrupt(0), now(2168732), total(3840276), url(https://rcavlive-dai.akamaized.net/hls/live/696614/cancbftprem/20190819T033438/master_5000/00010/master_5000_00935.ts)
E/STREAMING_ENGINE(  916): CHLSDataProcessMgr.cpp: StateError(852) > >> CHLSDataProcessMgr::StateError -53
E/STREAMING_ENGINE(  916): CHLSDataProcessMgr.cpp: StateError(880) > !! CHLSDataProcessMgr::StateError -53
E/STREAMING_ENGINE(  916): CHLSDataFetcher.cpp: LoadData(1457) > [Audio-Seg] Download next segment
E/WEBAPI_PLUGINS( 6326): avplay_adapter.cc: setErrorEventCBNative(2113) > Entered and the error_code = [-26476506]
E/WEBAPI_PLUGINS( 6326): avplay_adapter.cc: setErrorEventCBNative(2185) > PLAYER_ERROR_CONNECTION_FAILED
I/TIZEN_N_PLAYER( 6326): player.c: player_get_state(2572) > player address:[0xb62638]

Additional Information

Please include any additional information necessary here. Including the following:

versions

videojs

7.5.5

browsers

Unique problem to most Tizen TVs

OSes

Most Tizen TVs, however we have observed that with some firmware the issue does not entirely seem to be present on at least one of our Tizen TVs that we have tested on.

plugins

videojs-contrib-eme 3.5.0

[welcome[bot]]

👋 Thanks for opening your first issue here! 👋

If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.
To help make it easier for us to investigate your issue, please follow the contributing guidelines.

[kevinscroggins-youi]

I have also tried Video.js 7.6.3 with contrib-eme 3.5.4 with custom code to return false when checking if native MediaSource or audio/video/text tracks are supported, but this did not seem to yield any changes to the freezing.

[kevinscroggins-youi]

Additional investigation into other MSE players (Shaka, Bitmovin) and device logs shows that they do not use Tizen's broken streaming engine for video playback for either DASH or HLS VOD / Live. I am also able to override native tracks on Chrome, but it doesn't seem to let me do so on Tizen. There must be some way around this, perhaps something I'm missing. Native device logs when using Shaka / Bitmovin only reference the Chromium MSE player and the Tizen Native player:

I/TIZEN_N_PLAYER(12188): player.c: player_push_media_stream(4424) > player address:[0xc70d40]
I/CHROMIUM(12188): [INFO:media_source_player_capi.cc(981)] > Pushing media packet: timestamp: 76.9352s type: AUDIO data ptr: 0 data size: 476 shm handle: 171

[kevinscroggins-youi]

It appears that after some further troubleshooting, updating my player configuration to the following seems to do the trick, and the underlying Tizen streaming engine is no longer being used and the stream no longer freezes immediately:

            self.player = videojs(self.container.id, {
                controls: false,
                autoplay: false,
                loadingSpinner: false,
                html5: {
                    overrideNative: true,
                    nativeTextTracks: false,
                    nativeAudioTracks: false,
                    nativeVideoTracks: false
                }
            });

I think the big take-away from this is that the underlying native Tizen streaming interface is extremely broken, which is why we moved away from the native Tizen player to begin with. It's also worth noting that embedded metadata was never reported when running on a Tizen TV device as well. I would have to highly recommend that native tracks be disabled by default when running on a Tizen TV for the best experience possible when using Video.js. You can check either the user agent or search for webapis in the global namespace to check if you are on Tizen. More info here:
https://developer.samsung.com/tv/develop/guides/fundamentals/retrieving-platform-information
https://developer.samsung.com/tv/develop/api-references/samsung-product-api-references/webapi-api

[kevinscroggins-youi]

I'm also noticing that the audio seems to be garbled (at least for the current HLD live stream with AAC encoded audio that I am testing), and switching to use native audio tracks seems to resolve this without any freezing as well.

Additionally, SSAI ID3 metadata is not working either on Tizen, but that is probably a separate issue.

[gkatsev]

I was going to mention the native tracks stuff yesterday but got distracted by other things :(
Glad you were able to figure it out!
Yeah, unfortunately, a lot of native HLS/DASH on devices is really really really bad and MSE based playback is significantly better. Except on Safari/iOS, I would definitely recommend not using native. We've actually been thinking about just enabling that in VHS (@videojs/http-streaming, our playback engine) by default, but haven't had the chance to think about the ramifications of that.

Also, you should drop by our slack for more real time chat 🙂.

[kevinscroggins-youi]

Great, thanks for the response! Tizen is certainly no exception. When developing a TV application, the only option is to create an HTML5 application and with the built in AVPlay API being as awful as it is, it's often mandatory to implement an alternative MSE/EME based video player such as Video.js which is what led me here. We've had mixed results with some other MSE players such as Shaka and Bitmovin, but I have been much more impressed with Video.js so far. 🎉 I'll definitely drop by Slack! Thanks for mentioning. This ticket can probably be closed, unless you feel there should be any action taken to override the native text and video tracks on Tizen?

[gkatsev]

We probably should just enable overrideNative by default on all platforms except safari, but that's a pretty big change.
Let's close this. The issue will be around for people who have issues in the future and if we keep getting asks about it, we can add it to our FAQ or troubleshooting page.