Shaka-player: Clear Key problem : new implementation of a Common Encryption packager

[ut0mt8]

EDIT : it work both with shaka-player and dashjs in firefox. Not in chrome.
EDIT2 : implementing sub sample it work now both with shaka-player and dashjs in Chrome. Not in Firefox....
EDIT3 : it seems to work ! need some clarification and adjustment.

A bit of context first: (and sorry if this not really a shakaplayer issue, but I know there are very competent persons reading here). And sorry for the crosspost on the dashjs but the issue is generic...

I've started coding/adding common-encryption to nginx-rtmp module dash packager.
The reasons are :

• there a not so much live dash DRM (wdv, playready) enabled solution (USP have one but bases on smooth ingest which are ... old, elemental need to much module
• having an open source could be cool
• we already use a patched nginx-rtmp in production to handle client side ad insertion, which basically convert SCTE35 message to emsg box. It work well for us, so using something different for live streaming will need to rework with vendor this feature.

After some weeks reading the spec, and some weeks of coding I think I am not too far from success.
I can write encrypted fragment, made signalisation in init file. Decrypting it with mp4decrypt from bento4 tools (appending init and fragment file) result in a playable mp4. So my code should not be completely crap.

Code is here : https://github.com/ut0mt8/nginx-rtmp-module/tree/cenc/dash/

The next step is to test with clear key drm system (I will handle wdv and playready after if it work)
And obviously it do not work for now.

My test page is here : https://futomaki.net/player/shaka/
The manifest is here : https://aws.futomaki.net/dash/live/index.mpd

KID is

• F10DF10DF10DF10DF10DF10DF10DF10D (hex)
• 8Q3xDfEN8Q3xDfEN8Q3xDQ (base64)

KEY is

• F00DF00DF00DF00DF00DF00DF00DF00D (hex)
• 8A3wDfAN8A3wDfAN8A3wDQ (base64)

Observed behaviour

The content did not play :/

Console output

"Error: Shaka Error MEDIA.VIDEO_ERROR (3,,CHUNK_DEMUXER_ERROR_APPEND_FAILED: Failed to prepare video sampl

Any help, advices are welcome. If you notice something not conform or else don't hesitate ;)

[TheModMaker]

First, you could use Shaka Packager to generate your content. It takes UDP streams or files, supports Live and VOD, and supports clear-key and Widevine (and maybe PlayReady).

I don't notice anything obviously wrong with the content or the code, but common encryption is hard to get right. This is definitely not a Shaka Player issue and we can't offer official support. But I have worked on common encryption a lot recently, so I can take a look at it a bit.

What would really help would be a static media file before and after your encryption. I don't notice anything wrong with the container data, but without the original, it will be nearly impossible to tell where the encryption is wrong.

Also, OpenSSL EVP supports CTR, so in your encrypt method, you could just use EVP_aes_128_ctr and avoid writing your own CTR encryption. I would also highly advise checking the return value and w values to double-check it actually encrypted everything.

[ut0mt8]

Hello and thank for the response I really appreciate.
If you have worked on common encryption a lot I think you are the good person to help me.
Perhaps we could use another place to talk on that ?

1) I cannot use any existing packager (or I should patch) because we already use a patched version (by myself) of nginx-rtmp which handle scte35 message to pass it as emsg box. And this is very specific to our encoder (elemental). That said if something exist to repackage dash to encrypted dash on the fly I take.

This is why I'm begin to code it (and for the fun).

2) Having clear and encrypted media of the same fragment should be feasible, but this is not trivial.
The problem is that we are on live stream so, I have to patch my code to write both flavours. I was thinking of that from the beginning but the structure of the code does not make it simple.
So when done I can provide both init and fragment clear and encrypted.

3) I'm pretty sure it encrypt. When hexdumping I saw pure garbage(crypted stuff) in mdat. AND.

• the fragment was unplayable BUT decrypting it with mp4decrypt from bento4 tool with the good kid:key gave me a correct fragment playable with vlc ! this is why I think I have not mode complete crap. On the other side MP4Box cannot decrypt it correctly ?!

4) I will check how using EVP_aes_128_ctr (I think I have tried it and it did not make what I want but I could be wrong.) On the code my principal inspiration was the kaltura-vod-module and the ISO spec.

Couple of other questions :

• I did not make sub-sample encryption for video track for the moment. I know this is a Must in the spec but did that make a difference in the end ? I know that some packager (USP for example) did not that too, and content is playing.

• Did you know any other tool to test the conformity of my stuff ? how can I debug more precisely the browser side (EME, CDM, etc...) ?

Thank you.

[theodab]

Incidentally, your MPD has typos. On line 34, you are trying to close a ContentProtection element with <ContentProtection>. That should be </ContentProtection>. Same problem on line 71.

[TheModMaker]

1. Having clear and encrypted media of the same fragment should be feasible, but this is not trivial.
   The problem is that we are on live stream so, I have to patch my code to write both flavours. I was thinking of that from the beginning but the structure of the code does not make it simple.
   So when done I can provide both init and fragment clear and encrypted.

Does your code not support VOD? Couldn't you just tell it to encrypt some normal VOD clip and test with that instead? Do you have access to the source live stream? Since I assume your code isn't doing transcoding, all I really need is the codec data, which I could get out of the source TS stream.

Alternatively, you could set up a live stream based on a VOD clip. For example, this will stream the given MP4 file to a UDP stream, which could be picked up by your code.

ffmpeg -re -i input.mp4 -f mpegts udp://127.0.0.1:1234
# Or using rtmp
ffmpeg -re -i input.mp4 -f flv rtmp:///live/myStream.sdp

the fragment was unplayable BUT decrypting it with mp4decrypt from bento4 tool with the good kid:key gave me a correct fragment playable with vlc ! this is why I think I have not mode complete crap. On the other side MP4Box cannot decrypt it correctly ?!

When I try to decrypt a segment with mp4decrypt, the decrypted segment is still unplayable for me. I also can't decrypt it with FFmpeg. For example, if I run:

ffmpeg -decryption_key F00DF00DF00DF00DF00DF00DF00DF00D -i input.mp4 out.mp4

This will print a bunch of errors in NAL unit size, which is consistent with badly encrypted content. If I try to use FFmpeg to play the decrypted file, it gives the same errors.

• I did not make sub-sample encryption for video track for the moment. I know this is a Must in the spec but did that make a difference in the end ? I know that some packager (USP for example) did not that too, and content is playing.

I'm not sure if Chrome supports full-sample encryption for this. I know FFmpeg does, which is what I have been using for testing. But you should add subsample encryption later so it will be playable on the most platforms.

[ut0mt8]

@theodab thank for pointing it. I've fixed it. And also the content of the pssh box in base64 which it incorrect.

[ut0mt8]

@TheModMaker
No my code doesn't support vod but I already test streaming in rtmp from ffmepg , something like :
ffmpeg -re -fflags +genpts -stream_loop -1 -i video-fixed.mp4 -vcodec copy -acodec copy -f flv rtmp://localhost/live/live

This is the classic big buck bunny mp4, but with fixed GOP size at 50 frames (better for streaming).

So theoretically I can compare part of the initial file with resulted encrypted fragment. But I will try to generate clear fragment in the right order , to facilitate debugging.

for mp4decrypt : I've just retry on my local mac, so download init and one fragment, concat and it work.

$ wget https://aws.futomaki.net/dash/live/init.m4v
$ wget https://aws.futomaki.net/dash/live/526480.m4v
$ cat init.m4v 526480.m4v >> test-enc.m4v
$ ./Downloads/Bento4-SDK-1-5-1-623.universal-apple-macosx/bin/mp4decrypt --key 1:f00df00df00df00df00df00df00df00d test-enc.m4v test-dec.m4v

and VLC can play it.

On the ffmpeg side :

$ ./bin/ffmpeg -decryption_key F00DF00DF00DF00DF00DF00DF00DF00D -i test-enc.m4v test-ff.m4v
gave me no error, and it is also playable in VLC.
The only thing that is strange is that the resulting file is half the size, I imagine that ffmeg re-encode it.

Anyway this is encouraging no ? (on the other hand I don't understand why you didn't have the same result ?)

[TheModMaker]

First, you are using an older version of FFmpeg which incorrectly parses the encryption info. I sent in patches to fix it recently and am using that version, which is why I can't get it to work for me.

The problem is you use the saio box to indicate where the encryption info is, but the value is wrong. In older versions of FFmpeg, they didn't use this box, which is why it works in your FFmpeg and probably in mp4decrypt.

The number in the saio box is relative to the base offset defined in the tfhd (track-fragment header), which in your case is the moof. In your case, it appears to be off by 24 bytes (at least in the segment I checked).

[ut0mt8]

@TheModMaker Ah I was sure I made a mistake here.

So to my understanding the offset in saio should be the difference between the beginning of the moof box (not including the moof box header) and the first IV in senc box ?

So In my case I calculate it as is :

````
offset = (saio_pos - moof_pos) + saio_box_size (12 header/version/flags + 4 entry_count + 4 entry offset) + senc_size_header(12 header/version/flages + 4 sample_count).
`````

I miss something but what ? where is the 24 bytes missing ?

[TheModMaker]

I miss something but what ? where is the 24 bytes missing ?

https://github.com/ut0mt8/nginx-rtmp-module/blob/cenc/dash/ngx_rtmp_mp4.c#L1422

That should use moof_pos, not pos, which is the traf position.

[ut0mt8]

What a dumb :) good catch @TheModMaker !
The perfect typo. Difficult to find without an external eye..

Again thank you for the code review. I ve just fix it. (the code is bit crappy I know, I have to refactor/clean it).

Now it work with mp4decrypt, ffmpeg both version, MP4box ! We progress.

Still not work on any browser anyway (nor shaka or dashjs on chrome for example)

[TheModMaker]

I can get it to play in Firefox. My guess is Chrome doesn't support full-sample encryption.

[ut0mt8]

Yes it work both with shaka-player and dashjs in firefox.
And it doesn't at all in chrome; perhaps full-sample encryption for the video is not supported...
The sound track should be working independently ?
Did some Chrome-dev/other know that point ?

Anyway let's implement the norm and see.

[ut0mt8]

Update:

I've just implemented sub-sample encryption and now it work with Chrome but not on Firefox...

@TheModMaker what I was doing wrong this time. If you can keep an eye on it, your external viewpoint have been precious for now :)

Argh my saiz should be wrong : I will correct it.

[ut0mt8]

Quick status for followers and specially @TheModMaker.
So with sub-sample and an correct saiz box I ve got something which it close to work.

The only point is the size of the clear bytes at the beginning. Chrome seems to be very picky on that. For now I put an arbitrary big value.
Anyway this is almost safe as I encrypt sufficient amount of data to prohibit viewing anything.

[TheModMaker]

There are specific rules for what bytes need to be encrypted vs clear when using subsample encryption. Usually, decoders will just do what the container says and decode the bytes specified in the senc box, but Chrome may require you to follow those rules.

For example, in H.264/H.265, the data is written in a series of NAL units. Only video-slice NAL units should be encrypted and then the NAL header and the H.264/H.265 header should be kept in the clear. So if you want to follow the spec, you'll need to parse the NAL unit stream and the codec header to find which bytes to keep in the clear. You can check out the implementation in Shaka Packager for reference.

So short answer: Chrome probably requires you to keep certain parts in the clear.

Is there anything else you need help with?

[ut0mt8]

Again a big thanks for your support and patience.

The status is I have now a working implementation but still interoperability issue using clear key (I am in progress with widevine).

• Shaka player : Chrome OK ; Firefox OK
• Dashjs : Chrome NOK (?!) ; Firefox OK

PS : I knew about the norm about what to be crypted or not. (I got the ISO spec)
I've made a naive implementation , leaving enough arbitrary bytes in clear at the begining and it do the trick (assuming there only one NALU per frame; which it my case). This seems ok.

I think we could close here. I will continue on dashjs / chrome side.

[TheModMaker]

Glad I could help. If you have any more questions, feel free to ask.

[ut0mt8]

When I started I was thinking I ve never get it to work and I am a fool to try to implement it. And finaly even with widevine it work. Just before the Word cup :)

[joeyparrish]

:tada: Github calls that the "hooray" reaction, or the "tada" emoji, but I prefer to think of it as a vuvuzela. :tada: