Shaka-player: 608 Captions Data is not captured in HLS, Live

[arturparkhisenko]

Have you read the FAQ and checked for duplicate open issues?
Yes

What version of Shaka Player are you using?
v2.5.2

Can you reproduce the issue with our latest release version?
Yes (v2.5.9)

Can you reproduce the issue with the latest code from master?
Yes

Are you using the demo app or your own custom app?
Demo app

If custom app, can you reproduce the issue using our demo app?
N/A

What browser and OS are you using?
Google Chrome v80.0.3987.149 (Official Build) (64-bit), macOS Mojave 10.14.6

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
N/A

What are the manifest and license server URIs?
The URL doesn’t have long TTL, if necessary, I could provide it separately.

What did you do?
Copied the manifest URL, did setup the custom content in the Demo application, click play if it does not autoplay.

What did you expect to happen?
I see the captions menu with the ability to select from or turn them off.

What actually happened?
Embedded Captions doesn’t work, there is no option to select the captions in Demo application.

Looks like Media Source Engine pumps data to the Text Engine. I was able to identify that mux.js parses caption data from the stream. But in the text engine there is a compromised comparison for identifiers. Also, it impacts the Streaming Engine update routine where the MIME type for embedded captions is not explicitly supported.

[michellezhuogg]

@arturparkhisenko ,
Thank you for reaching out to us! Could you please share your manifest with us, so that we can reproduce the issue?

[arturparkhisenko]

@michellezhuogg do you have an email where I can send the URL?

[michellezhuogg]

[email protected]
Thanks!

[arturparkhisenko]

@michellezhuogg I sent the email! Thanks. I've also done a re-test for v2.5.10, and it's still reproducible.

[arturparkhisenko]

@michellezhuogg did the URL we have provided work?

I've discovered a good overview of different Closed Captioning Samples, I'm curious if Transport Streams (TS) and any of that insertion ways will be planned for the shaka-packager?

[michellezhuogg]

Hello @arturparkhisenko ,

1. I looked into the url you shared with us in the email, and was able to see the captions. Could you please specify what captions were you expecting but not displaying?
2. Could you please share a url for a manifest that is not behind a paywall? It would be much easier for us to reproduce and test directly on our demo app.
   Thank you!

[arturparkhisenko]

@michellezhuogg sorry, didn't know that paywall was enabled after I shared the URL.
It was hard to find the content to reproduce, but I found the steps with publicly available content.

Steps to reproduce it with VOD files (not specific to Live):

1. Download the test sample file from the CCextractor site MPG, 608 and 708, SCTE 128 / DTVCC Transport
2. With a power of FFmpeg it's possible to generate master manifest with that command (replace the video.mpg to the name of the file you downloaded):

ffmpeg -i video.mpg -f hls -master_pl_name master.m3u8 chunks.m3u8

1. Serve it and load as custom content in the Shaka-Player.

How I did test that:

1. Checkout Shaka Player v2.5.10 and did run npm i && ./build/all.py
2. Moved the FFmpeg output (master.m3u8, chunks.m3u8, chunks0.ts...) to the shaka-player/demo/ folder
3. Did run python -m SimpleHTTPServer 8080 in the shaka-player root directory.
4. Navigated to the http://localhost:8080/demo/ and created a new custom content with a hosted master playlist URL http://localhost:8080/demo/master.m3u8 or just http://localhost:8080/demo/#audiolang=en;textlang=en;uilang=en;asset=http://localhost:8080/demo/master.m3u8;panel=CUSTOM%20CONTENT;build=uncompiled

_Note 1._ This test content has open captions, I suppose for the test purposes, but we're looking for closed captions detection in the player and ability to detect, select them and show.
_Note 2._ I did test that captions are existed and extracted by mux.js using it's debug/captions.html page by uploading generate *.ts files from the steps above.

[michellezhuogg]

@arturparkhisenko , thank you for providing the manifest!

For captions embedded in the video, f the playlist doesn't contain CLOSED-CAPTIONS metadata, we can't know if there's captions or not until the video starts playing and the data gets transmuxed. Thus, we might not be able to show the captions on/off options in the menu.
If you have a manifest with CLOSED-CAPTIONS attributes in the playlist, it's expected to see the captions options displayed in the menu.

If you want us to detect and display embedded closed captions data on-the-fly, that might be the same issue as https://github.com/google/shaka-player/issues/1826 .

[arturparkhisenko]

@michellezhuogg thank you. We can close this ticket then. While there are cases where it's possible to detect embedded closed captions, I think there are cases around live broadcast where it should be detected on the fly. It would be nice if we could configure behavior for detection on the fly. For example, it could be a configuration property like alwaysScanForClosedCaptions similar to the one which we already have alwaysStreamText.

manifest with CLOSED-CAPTIONS attributes in the playlist

I can confirm that CLOSED-CAPTIONS attribute is working. This is an example of the playlist tags I used for testing:

#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="cc",LANGUAGE="eng",NAME="English",AUTOSELECT=YES,INSTREAM-ID="CC1"

#EXT-X-STREAM-INF:BANDWIDTH=140800,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2",CLOSED-CAPTIONS="cc"
https://my-streaming-service.com/manifest/level/1080.m3u8

[joeyparrish]

Okay, then we'll close this issue. We'd be happy to take a PR for #1826, though. Thanks!