Dash.js: How do I NOT display webvtt listed in mpd as caption?

[ddhp]

Hello there!

We are trying to list a text/vtt adaptation inside mpd, and do not want it to be displayed as caption, it's more like a metadata instead.

And we found out that dash.js treats mimeType: text/vtt adaptation as a caption kind, if it's role is not set to caption or subtitle.

ref: src/streaming/TextSourceBuffer.js:152

var trackKindMap = { subtitle: 'subtitles', caption: 'captions' };
var getKind = function () {
                var kind = (mediaInfo.roles.length > 0) ? trackKindMap[mediaInfo.roles[0]] : trackKindMap.caption;
                kind = (kind === trackKindMap.caption || kind === trackKindMap.subtitle) ? kind : trackKindMap.caption;
                return kind;
            };

Can we add metadata inside trackKindMap?

Thanks!

[wilaw]

@Chien-Chen Chen - this makes sense. I opened a new feature enhancement issue https://github.com/Dash-Industry-Forum/dash.js/issues/1238 on your behalf.

Cheers

Will

From: Chien-Chen Chen <[email protected][e-mail]>
Reply-To: "Dash-Industry-Forum/dash.js" <[email protected][e-mail]>
Date: Sunday, March 6, 2016 at 8:09 PM
To: "Dash-Industry-Forum/dash.js" [email protected][e-mail]>
Subject: [dash.js] How do I NOT display webvtt listed in mpd as caption? (#1236)

Hello there!

We are trying to list a text/vtt adaptation inside mpd, and do not want it to be displayed as caption, it's more like a metadata instead.

And we found out that dash.js treats mimeType: text/vtt adaptation as a caption kind, if it's role is not set to caption or subtitle.

ref: src/streaming/TextSourceBuffer.js:152

var trackKindMap = { subtitle: 'subtitles', caption: 'captions' };
var getKind = function () {
var kind = (mediaInfo.roles.length > 0) ? trackKindMap[mediaInfo.roles[0]] : trackKindMap.caption;
kind = (kind === trackKindMap.caption || kind === trackKindMap.subtitle) ? kind : trackKindMap.caption;
return kind;
};

Can we add metadata inside trackKindMap?

Thanks!

—
Reply to this email directly or view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Dash-2DIndustry-2DForum_dash.js_issues_1236&d=CwMCaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=iG1fmIm__as9N1xa_yLsAHRuda9SP_4lRAjEfvJHDGU&s=s8w23ovocsX_6SZFHg-QRIqAY0Tvwq-Do4eZksVDioA&e=.

[davemevans]

I can see the problem here but I have a couple of concerns.

The Role descriptor is not mandatory and I think the most sensible option is to default to caption in the absence of any other signalling. Role does not define metadata as an acceptable value for @value at least in the spec version I have [1], so it's not obvious what value it would map to in trackKindMap?

[1] Table 22, ISO 23009-1 2014

[JibberJim]

The TextSourceBuffer doesn't appear to acknowledge or do anything with the
@schemeIdUri, which is how this should presumably be handled (with a self
defined Uri and whatever they want) is this a general case problem in
dash.js - that it just assumes everything is in the "normal" uri?

ie

should be something like

And that should be supported by in the spec, and a suitable default for a
text track is actually metadata for an unknown role (as opposed to an
unspecified one), since that is more likely to map correctly, as we know
it's unlikely to be captions or subs as there are existing identifiers for
that?

Jim.

On Tue, Mar 8, 2016 at 2:28 PM, David Evans [email protected]
wrote:

I can see the problem here but I have a couple of concerns.

The Role descriptor is not mandatory and I think the most sensible option
is to default to caption in the absence of any other signalling. Role does
not define metadata as an acceptable value for @value at least in the
spec version I have [1], so it's not obvious what value it would map to in
trackKindMap?

[1] Table 22, ISO 23009-1 2014

—
Reply to this email directly or view it on GitHub
https://github.com/Dash-Industry-Forum/dash.js/issues/1236#issuecomment-193803763
.

[ddhp]

Hello @bbcrddave ,

Understand your concern.
Here kind is an attribute of HTML5 TextTrack dom element, according to MDN doc:

kind is How the text track is meant to be used. If omitted the default kind is subtitles. The following keywords are allowed:

• subtitles
• captions
• descriptions
• chapters
• metadata

That's why I assume that trackKindMap supposed to list metadata :p

And you are right, there is no such role inside DASH mpd spec, so the problem might be:「how to map adaptation role to html5 TextTrack kind attribute」

Thanks!

[davemevans]

I've retracted my previous comment since HBBTV only deals with TTML, not VTT so isn't really relevant. There's an unofficial W3C draft which says similar (https://dev.w3.org/html5/html-sourcing-inband-tracks/), but I'm not convinced it's really the right answer.

I think that the problem is that there isn't a way to do this mapping using just the MPEG spec. @JibberJim's suggestions of using an unknown, private schemeIdUri is possibly the best option.

Other options are to use the MPEG-defined way of delivering metadata (EventStreams), or to add an EssentialProperty descriptor (not currently supported in dash.js) which would instruct the player to ignore the adaptation set entirely though I assume you are expecting the player to render the data into cues on a track for you so this probably wouldn't work.

[TobbeEdgeware]

From a DASH point of view, I think emsg is the proper tool.

As was already evident from the discussion above, the kind property of HTML-5 tracks does not map to DASH roles.

We have the mapping between DASH roles and HTML5 track kinds

subtitle -> subtitles
caption -> captions

but no role in DASH that is close to metadata.
I don't think we should implement a private schemeIdUri in dash.js since it is a reference player.
One can possibly try to add the metadata role in a future version of DASH.

[TobbeEdgeware]

PR #1801 lets the application get the "role" value, so that it can do something with it like filtering out certain tracks. I close this for now.