Shaka-player: Question: mixing encrypted with unencrypted multi period content

[gesinger]

We tried looking for some older issues/commits/docs, but couldn't find a reference, so figured we'd open an issue to start a discussion (if you think this is best put elsewhere, feel free to close).

Mainly we wanted to ask about how you handle multi period content where you start with unencrypted content and follow it with encrypted content.

This morning we submitted https://bugs.chromium.org/p/chromium/issues/detail?id=895449 noticing that if you simply append encrypted content after unencrypted, it will crash the tab in Chrome (though it will play through in Firefox). The reverse, appending unencrypted after encrypted, works fine. There's a reduced test case in that example which exhibits the behavior.

However, we noticed that Shaka doesn't exhibit such behavior, and manages to play through just fine, even on Chrome. We looked at some of the differences in behavior, but couldn't find anything which directly helped the case we posted in the Chromium bug, though we could be missing something else.

We were wondering if you had any thoughts on what might allow Shaka to play through the content, to work around the browser's behavior.

Thanks in advance.

[vaage]

Right now when we see unencrypted content, we attempt to negotiate key system access for all the common key systems. That way eme is already set up when encrypted data arrives. This works most of the time as long as the codecs don't change - which is what #1567 is about.

If we successfully negotiated access to a key system, we pass all content (even unecrypted content) through EME so that if it become encrypted we are still able to play it. If we could not negotiate access, then we play the content with out EME and will crash if it becomes encrypted again later.

Right now the code for this is in DrmEngine.queryMediaKeys. We realize that part of the code is not the easiest the read and I am personally exploring different ways to structure it so that it will be easier to read in the future.

We are currently working on ways to better support scenarios like #1567. Needless to say it is a very complex issue and is something that @joeyparrish and I are actively talking about ways to handle this.

Does that help?

[gesinger]

Thank you for the quick and detailed response @vaage , we appreciate it.

We gave it a shot (creating the media keys and setting them on the video element before appending any content), and it plays through now. We'll use that strategy for the time being and see if the Chrome team will add support for setting media keys when needed, rather than in advance. We'll update that Chromium bug with that note.

Again, thank you for the help!

[joeyparrish]

Just so you know, I would support changes in the EME spec that would actually require MediaKeys to be set up before any content is appended to SourceBuffers. I think this is just the right way to do things at the app level.

I know that at least within Chrome, there are different media stack paths based on whether content is encrypted or not. So it may not be practical for all browsers to allow you to append content first and set up MediaKeys later.

I hope that context is helpful.

[gesinger]

Thanks for the additional comment @joeyparrish .

In the case of live with mixed content, where you may not be sure ahead of time what content types you will have and whether they are encrypted or not, couldn't this solution potentially run into problematic situations with the underlying browser media stack paths?

[joeyparrish]

How do you not know your content could be encrypted? We don't know, because we're middleware, so we have a way to configure us to expect it. If you're an end-user app, we would expect you to know what your content looks like. If that's a bad assumption, please tell us!

It's also worth noting that you can always play clear content through an encrypted path, so there's no major problem with setting up MediaKeys if you don't need it.

My view is that if it's VOD, you know everything about it in advance, because it's static. If it's live, you may not control the ads that are played, but they are more likely to be in the clear than encrypted. It's your main content that's more likely to be encrypted, and that's the part the streaming service provider controls and knows about. There's no sense in encrypted ads on clear content.

And if you're making middleware like we are, you should probably leave it up to the app developer to decide if we should set up MediaKeys.

Our convention is that if there's a license server, you should set up MediaKeys. You can't use MediaKeys without one, so that makes sense for us. I'm totally open to another means to configure that if it doesn't suit you.

Does that help?

[gesinger]

(Sorry for the delay, and for responding on a closed issue, just wanted to follow up.)

I was thinking about a case where the content could go from unencrypted ad to encrypted content in a case like HLS, where the manifest is refreshing and the encrypted content only appears after a refresh or two. But you're right that content creators should know if the content is encrypted or not, and when the player is set up the license server should be provided.

Playing clear content through an encrypted path does seem reasonable enough, and that seems to be the best approach here. Using the presence of a license server as a guide of whether or not to set up that path also seems the best decider for this case.

I think it all seems like a good approach, and I appreciate the detail. This definitely helped. Thank you again!

[joeyparrish]

I'm glad we could help!