Shaka-player: Support CEA-608/708 on DASH

Created on 11 Apr 2018 · 30Comments · Source: google/shaka-player

Have you read the FAQ and checked for duplicate open issues?: Yes

What version of Shaka Player are you using?: 2.3.5

Can you reproduce the issue with our latest release version?: Yes

Can you reproduce the issue with the latest code from master?: Yes

Are you using the demo app or your own custom app?: Demo app

What browser and OS are you using?: Amy

What are the manifest and license server URIs?:
VOD CEA-608:
https://vm2.dashif.org/dash/vod/testpic_2s/cea608.mpd
Live CEA-608:
https://vm2.dashif.org/livesim/testpic_2s/cea608.mpd
VOD CEA-608 + TTML
https://vm2.dashif.org/dash/vod/testpic_2s/cea608_and_segs.mpd
Live CEA-608 + TTML
https://vm2.dashif.org/livesim/testpic_2s/cea608_and_segs.mpd

According to https://dashif.org/docs/DASH-IF-IOP-v4.2-clean.pdf

6.4.3.3. Signaling CEA-608 caption service metadata

The Accessibility descriptor shall be provided with @schemeIdUri set to urn:scte:dash:cc:cea-608:2015, and an optional @value attribute to describe the captions. If the @value attribute is not present, the Representation contains a CEA-608 based closed captioning service.
DASH-IF Interoperability Points v4.1 141
If present, the @value attribute shall contain a description of caption service(s) provided in the stream as a list of channel-language pairs. Alternatively, a simple list of language codes may be provided, but this is strongly discouraged as it will not provide sufficient information to map the language with the appropriate caption channel.
The @value syntax shall be as described in the ABNF below.
@value = (channel *3 [";" channel]) / (language *3[";" language])
channel = channel-number "=" language
channel-number = CC1 | CC2 | CC3 | CC4
language = 3ALPHA ; language code per ISO 639.2/B

Examples:
Simple signaling of presence of CEA-608 based closed caption service (Note: Not signaling languages is a discouraged practice)
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015"/>
Signaling of presence of CEA-608 closed caption service languages in English and German
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015" value="eng;deu"/>
Signaling of presence of CEA-608 closed caption service in English and German, with channel assignments
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-608:2015" value="CC1=eng;CC3=deu"/>
Signaling of presence of CEA-708 closed caption service in English and German
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-708:2015" value="1=lang:eng;2=lang:deu"/>
Signaling of presence of CEA-708 closed caption service in English and easy reader English
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-708:2015" value="1=lang:eng;2=lang:eng,war:1,er:1"/>

Note: In DASH-IF player, it works (http://reference.dashif.org/dash.js/v2.6.7/samples/dash-if-reference-player/index.html)

archived enhancement

Source

avelad

All 30 comments

Sample of cea608 parser: https://github.com/Dash-Industry-Forum/dash.js/blob/development/externals/cea608-parser.js

avelad on 11 Apr 2018

We currently only support CEA608/708 in TS segments, so we would need to investigate how to extract them from MP4 segments. Also, our current CEA608/708 support is based on the parser provided by mux.js. We would need to investigate whether or not it makes sense for mux.js to do this for MP4, as well.

joeyparrish on 11 Apr 2018

We're actually planning this quarter on parsing fmp4 608/708 in mux.js in order to handle fmp4 HLS and DASH. Stay tuned!

forbesjo on 11 Apr 2018

@forbesjo, Do you have in mind to support multiple CC in the same MP4?

avelad on 13 Apr 2018

@avelad yes that should be supported

forbesjo on 13 Apr 2018

Blocked on https://github.com/videojs/mux.js/issues/190

joeyparrish on 17 Apr 2018

@avelad @joeyparrish Are either of you aware of whether DASH specifies how subtitles should be prioritized, if DASH content has both CEA-608/708 in the video stream, but also side-loaded text tracks?

I would assume that the presence of side-loaded text tracks means that they should be prioritized over those embedded in the streams. But section 6.4.5 of IOP (Guidelines for side-loaded TTML and WebVTT files) does not specify.

chrisfillmore on 18 Apr 2018

@chrisfillmore, I'm not sure of the answer, but in Shaka Player, we give applications control over track selection.

joeyparrish on 18 Apr 2018

@joeyparrish Hi Joey been a while hope all is well? Wondering if you guys are looking at https://github.com/Dash-Industry-Forum/cea608.js for implementation. HLS.js used this and of course we used it for Dash.js. Feel free to use this if it suites your needs!

Update
I guess I overlooked the mention on that url so you are aware. just wanted to make sure!

dsparacio on 4 May 2018

@dsparacio, thanks for the link! We'll keep it mind if we decide to explore alternatives to mux.js.

joeyparrish on 7 May 2018

This is the mux.js PR for reference https://github.com/videojs/mux.js/pull/197

forbesjo on 2 Jul 2018

mux.js has released version 4.5.0 with support for CEA608 in mp4.

avelad on 17 Jul 2018

@joeyparrish - on which shake player version supporting CEA608 in mp4 will be available?

ronenneumark on 22 Jul 2018

Second that on 608 on mp4. We have many use cases for pulling captions in band.

rbond on 27 Jul 2018

@rbond - can you share the supported in band cases?

ronenneumark on 29 Jul 2018

Hi all,

When we update mux.js, there will probably still be integration work to complete on our side. As a new feature, this would not appear until v2.5 at the earliest.

joeyparrish on 2 Aug 2018

Just want to leave a +1 on this one. We have two features we really hope make it into Shaka 2.5 and this is one of them. -- The other is the support for key rotation in the manifest.

TheJohnBowers on 22 Oct 2018

Hi everyone,

We are hard at work on this, and it is a priority for us for v2.5. As soon as we have something to show for it, we will release another v2.5 beta. The final v2.5.0 will wait on both this and src= support (which enables both progressive mp4 playback and FairPlay support on macOS and iOS).

joeyparrish on 29 Oct 2018

I see that the issue is on backlog. @joeyparrish , can you change it to 2.5?

avelad on 29 Oct 2018

Which issue?

joeyparrish on 29 Oct 2018

@joeyparrish This one! :)
@avelad Let me do it :)

ismena on 29 Oct 2018

I have another live test stream that you can validate against here if it is useful:
manifest url: https://content.downlynk.com/channel/34b1b0717d9d44f6809c89ab130a16bf.mpd
license server: https://content.downlynk.com/wv

TheJohnBowers on 29 Oct 2018

❤1

Thanks, @ismena! I miss the obvious sometimes. And thanks, @TheJohnBowers, for the content! Very helpful.

joeyparrish on 29 Oct 2018

The work we've done so far has been released in v2.5.0-beta2. There are still integration tests to write, plus we need to align our HLS CEA implementation with the DASH CEA implementation, and we need to deprecate one-off APIs for HLS CEA in favor of the tracks API we are using for DASH CEA.

joeyparrish on 9 Nov 2018

@joeyparrish the configuration forceTransmuxTS (https://shaka-player-demo.appspot.com/docs/api/shakaExtern.html#.StreamingConfiguration) will be necessary after changes in your HLS parser?

avelad on 27 Nov 2018

Just tried to compile shaka player from master and failing to use generated shaka-player.compiled.js due to ReferenceError: muxjs is not defined. I know I could link mux.js library but shouldn't it work without the library? Just trying to play regular dash without closed captions.

I think the issue was introduced here https://github.com/google/shaka-player/commit/bcd10c272ea16334432583d5db725883078899f4#diff-7fe7cffd0a8a1684650ad9d4ef4afdbaR62

jakubvojacek on 18 Dec 2018

@jakubvojacek, you are right, mux.js should not be required. I just spoke with @michellezhuogg and we have a plan to address the issue and she is working on implementing it right now. Hopefully this won't not be an issue for much longer.