Shaka-player: Unsupported HLS KEYFORMAT com.apple.streamkingkeydelivery

[alexanderturinske]

Have you read the FAQ and checked for duplicate open issues?:
YES

What version of Shaka Player are you using?:
2.5.4

Can you reproduce the issue with our latest release version?:
YES

Can you reproduce the issue with the latest code from master?:
YES

Are you using the demo app or your own custom app?:
NO

If custom app, can you reproduce the issue using our demo app?:

What browser and OS are you using?:
Safari 12.1.2 on macOS 10.14.6

What are the manifest and license server URIs?:
manifest: https://.../manifests/hls/master.m3u8
license: ${this.baseUrl}/items${id}/drm/fairplay

What did you do?
Tried to play normal HLS

What did you expect to happen?
Video should be played

What actually happened?
The documentation seems pretty minimal as the support was just added, but this is what I have thus far

Configuration

player.configure({
  drm: {
    servers: {
      'com.widevine.alpha': `${this.baseUrl}/items/${id}/drm/widevine`,
      'com.microsoft.playready': `${this.baseUrl}/items/${id}/drm/playready`,
      'com.apple.fps.1_0': `${this.baseUrl}/items${id}/drm/fairplay`,
    },
    advanced: {
      'com.apple.fps.1_0': {
        serverCertificate: fairplayUtils.fetchCertificate(),
      },
    },
  },
});
...
load(streamUrl) {
  const url = 'https://.../manifests/hls/master.m3u8'
  player
    .load(url, null, shaka.hls.HlsParser)
    .then(() => { ... }
    .catch((e) => { ... }

I am passing in the master.m3u8 into the load function and get the following error

Error from Shaka Player

Error {
  category: 4
  code: 4026
  data: [] (0)
  handled: false
  message: "Shaka Error MANIFEST.HLS_KEYFORMATS_NOT_SUPPORTED ()"
  severity: 2
  stack: ...
}

Looking at shaka player's errors (honestly the best error codes I have seen from a third party)
it says HLS_KEYFORMATS_NOT_SUPPORTED | 4026 | The HLS parser has encountered encrypted content with unsupported KEYFORMAT attributes.

I then modified my webpack to use the debug version of shaka-player (config.resolve.alias = { 'shaka-player': require.resolve('shaka-player/dist/shaka-player.compiled.debug.js'), }) and modify the imports (import { shaka } from 'shaka-player';) and get the additional errors

[Warning] Unsupported HLS KEYFORMAT – "com.apple.streamingkeydelivery" (0fd27489a42d02c11796.js, line 209516)
[Warning] The walker saw an error: (0fd27489a42d02c11796.js, line 209812)
[Warning] Error Code: – 4026 (0fd27489a42d02c11796.js, line 209813)

The master.m3u8 returns a few prog_index.m3u8 endpoints and the highest resolution of them gets called. The response from: manifests/hls/...stuff.../prog_index.m3u8 endpoint is where the com.apple.streamingkeydelivery is coming from with #EXT-X-KEY

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-KEY:KEYFORMAT="com.apple.streamingkeydelivery",METHOD=SAMPLE-AES,URI="..."
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:10
#EXT-X-MAP:BYTERANGE="857@0",URI=".../...mp4"
#EXTINF:9.57867,
#EXT-X-BYTERANGE:160142@...//...mp4
#EXT-X-ENDLIST

Questions
1) Is the KEYFORMAT wrong in this response?
2) In Issue #896, one response was that SAMPLE-AES was not supported with widevine, is it supported with HLS?
3) Is there any additional configuration that needs to be done to support this KEYFORMAT?
4) Is this perhaps related to #1625 and/or #850 ? Though it says it is AES, not AES-128, but I haven't looked up the difference yet

Relates to #382 ?

Any help is appreciated! Thanks!

[TheModMaker]

We only support FairPlay through native src= playback, we don't support it through MSE playback. If you don't pass the third argument to load, we'll default to using src= and it should work fine. You should also look at the new FairPlay tutorial if you are having any trouble setting up src= playback.

[alexanderturinske]

Thanks!

Unfortunately, on modifying my load call to .load(url)that gets me back to the error I was hoping to fix by switching to shaka-player 🤦‍

{
  category: 6
  code: 6006
  data: Array (3)
    0 "EME PatchedMediaKeysApple key error"
    1 Error: EME PatchedMediaKeysApple key error
      column: 162
      errorCode: WebKitMediaKeyError {code: 1, systemCode: 0, MEDIA_KEYERR_UNKNOWN: 1, ... }
      line: 210074
      message: "EME PatchedMediaKeysApple key error"
      sourceURL: "https://....com:3000/256404836e07da8a76d4.js"
      stack: "onWebkitKeyError_@https://....com:3000
  2 undefined
handled: false
message: "Shaka Error DRM.FAILED_TO_GENERATE_LICENSE_REQUEST (EME PatchedMediaKeysApple key error,Error: EME PatchedMediaKeysApple key error,)"
severity: 2
stack: ...
}

Switching to shaka-player was not completely a waste though, the better error handling at least gave me some more information about the issue. I knew it was an EME issue, but now I have a few more things to search for.

From the docs

FAILED_TO_GENERATE_LICENSE_REQUEST | 6006 | number | The CDM was unable to generate a license request for the init data it was given.  The init data may be malformed or in an unsupported format.  error.data[0] is an error message string from the browser.  error.data[1] is the error object from the browser.  error.data[2] is a string with the extended error code, if available.

Additional errors being thrown by shaka

[Error] Invalid config, unrecognized key .streaming.abr
    mergeConfigObjects (48aafc6fd6e084d8d03d.js:209791:225)
[Error] Invalid config, unrecognized key .streaming.drm
    mergeConfigObjects (48aafc6fd6e084d8d03d.js:209791:225)
[Error] Invalid config, wrong type for .drm.advanced.com.apple.fps.1_0.serverCertificate
    mergeConfigObjects (48aafc6fd6e084d8d03d.js:209791)
[Error] Assertion Failed: missing certificate!
    onWebkitNeedKey_ (48aafc6fd6e084d8d03d.js:210071:368)
[Error] TypeError: null is not an object (evaluating 'b.byteLength')
    rebuildInitData_ (48aafc6fd6e084d8d03d.js:210069:383)
    onWebkitNeedKey_ (48aafc6fd6e084d8d03d.js:210071:481)

Seems like something wrong with my certificate, which is potentially causing an issue with rebuilding the initData

Any pro-tips on this one? 😄 😭

[TheModMaker]

The server certificate needs to be a Uint8Array. So if you are using XMLHttpRequest with an arraybuffer type or using fetch, then the result is an ArrayBuffer. You may need to wrap the result in a Uint8Array:

const req = await fetch('https://example.com/cert.der');
const cert = await req.arrayBuffer();

player.configure({
  drm: {
    advanced: {
      'com.apple.fps.1_0': {serverCertificate: new Uint8Array(cert)},
    },
  }
});

[alexanderturinske]

Thanks for the suggestion; I've got that going on.

player.configure({
  drm: {
    servers: {
      'com.widevine.alpha': `${this.baseUrl}/items/${id}/drm/widevine`,
      'com.microsoft.playready': `${this.baseUrl}/items/${id}/drm/playready`,
      'com.apple.fps.1_0': `${this.baseUrl}/items${id}/drm/fairplay`,
    },
    advanced: {
      'com.apple.fps.1_0': {
        serverCertificate: fetchCertificate(),
      },
    },
  },
});
...
fetchCertificate() {
  const cachedCert = certificateCache.get(this) && certificateCache.get(this).cert;
  if (cachedCert) {
    return cachedCert;
  } else {
    return axios({
      method: 'get',
      url: this.serverCertificatePath,
      responseType: 'arraybuffer',
      headers: this.headers,
    })
    .then(response => {
      const cert = new Uint8Array(response.data);
      certificateCache.set(this, { cert });
      return cert;
    })
    .catch(err => {
      console.log('fetchCertificate: err: ', err);
    })
  }
}

And I have verified that the cert is coming back as an array buffer.

[TheModMaker]

Your fetchCertificate method returns a Promise. You need to wait for it to complete. For example:

player.configure({
  drm: {
    advanced: {
      'com.apple.fps.1_0': {
        serverCertificate: await fetchCertificate(),
      },
    },
  },
});
// Or
fetchCertificate().then((cert) => {
  player.configure({
    drm: {
      advanced: {
        'com.apple.fps.1_0': {
          serverCertificate: cert,
        },
      },
    },
  });
});

[alexanderturinske]

Yup, you are absolutely correct. After fixing that, some errors went away, but ultimately I ended up with the same error.

I continued debugging and saw that our DRM license request was failing, and found out that that the url needed to go from ${this.baseUrl}/items${id}/drm/fairplay to ${this.baseUrl}/items${id}/drm/fairplay/license.

Now the DRM is receiving the response, but it giving me an error. 😑

In any case, the Shaka Error DRM.FAILED_TO_GENERATE_LICENSE_REQUEST error message really helped me realize that this is all a licensing request issue and that has been great.

Thanks for your help. I'll keep you posted on my progress.

[TheModMaker]

If you are getting DRM.LICENSE_RESPONSE_REJECTED, then one likely problem is your license server is wrapping the response. You may need to unwrap the response before it is passed to the browser. You can read more about that in our tutorial.

[alexanderturinske]

Sorry for the late response; I got pulled into some fires.

Nah, I am still getting the same error (Shaka Error DRM.FAILED_TO_GENERATE_LICENSE_REQUEST (EME PatchedMediaKeysApple key error,Error: EME PatchedMediaKeysApple key error,), and now our license server is erroring with
FPSUserException: SPC byte array too short! Cannot contain payload!, so I will look through your tutorial and see where I messed up.

[alexanderturinske]

Thanks for the tutorial.

In our original implementation, the other library we were using just gave us the WebKitMediaKeyMessageEvent.message, which is a Uint8Array, and then we converted it to base64Encoded and sent it to our license server.

In looking into the shaka-player code, I see that shaka-player does not simply append the WebKitMediaKeyMessageEvent.message to the request.body.

I implemented a request filter and was able to take a look at what the request body looked like and I see that is not a Uint8Array and if I convert it to one, it doesn't match what I am seeing on the WebKitMediaKeyMessageEvent.message from the debugger.

Looking at the Fairplay tutorial, I saw that in the example the filter was adding some additional information to the request.body, so fiddling with that, I found that the request.body has the extra spc= prepended to it, so I converted the request.body to the original Uint8Array via the following

const string = StringUtils.fromUTF8(request.body); //-> "spc=AAA..."
const withoutPrepend = string.substring(4); //-> "AAA..."
const originalUint8array = Uint8ArrayUtils.fromBase64(sub); //-> Uint8 [ 0 ....]

and that satisfied my license server.

I am still getting the DRM.FAILED_TO_GENERATE_LICENSE_REQUEST error, but now the license request is returning a 200 most of the time, so that's good. 🚀

Actually, I can get playback on the first clip I play, the first time I play it, but I still get the error

category: 6, code: 6006, WebKitMediaKeyError {code: 1, systemCode: 0}, message: "EME PatchedMediaKeysApple key error"

before the response filter is hit. The only errors I see are way before this one

[Error] Invalid config, unrecognized key .streaming.abr
[Error] Invalid config, unrecognized key .streaming.drm

Could those attribute to this error?

[TheModMaker]

Looking at the Fairplay tutorial, I saw that in the example the filter was adding some additional information to the request.body, so fiddling with that, I found that the request.body has the extra spc= prepended to it, so I converted the request.body to the original Uint8Array via the following

That is because you are probably copying the FairPlay tutorial, which has a call to player.getNetworkingEngine().registerRequestFilter() which converts the body to that URL. You could just convert that filter to do what you want.

Actually, I can get playback on the first clip I play, the first time I play it, but I still get the error
category: 6, code: 6006, WebKitMediaKeyError {code: 1, systemCode: 0}, message: "EME PatchedMediaKeysApple key error"

This may be #2031. You could try changing the error listener to ignore the error and see if you get playback.

before the response filter is hit. The only errors I see are way before this one
[Error] Invalid config, unrecognized key .streaming.abr
[Error] Invalid config, unrecognized key .streaming.drm
Could those attribute to this error?

Those are warnings from Shaka Player about invalid configuration paths. The abr configuration should be at the top-level, not part of streaming:

player.configure({
  abr: ...
});

[alexanderturinske]

Ah! #2031 is the github issue I have been looking for this entire time. I am converting from videojs-contrib-eme because of this issue.

That is because you are probably copying the FairPlay tutorial, which has a call to player.getNetworkingEngine().registerRequestFilter() which converts the body to that URL. You could just convert that filter to do what you want.

Your commit to remove the automatic appending of spc= didn't go in until ~Aug 5 and there hasn't been a release since before then, so I think this is just something I have to deal with until the next release. That, or the 2.5.4 is shipping the wrong shaka-player.compiled.js 🙃

This may be #2031. You could try changing the error listener to ignore the error and see if you get playback.

You are suggesting forking shaka player and making the change myself, correct? Or is there an external API of shaka-player's that I can use to turn it off?

Those are warnings from Shaka Player about invalid configuration paths. The abr configuration should be at the top-level, not part of streaming:

You are correct; I found some setting of the configuration that I hadn't seen before. A quick global search uncovered it.

Thanks again for all of your help; I really appreciate it.

[TheModMaker]

Your commit to remove the automatic appending of spc= didn't go in until ~Aug 5 and there hasn't been a release since before then, so I think this is just something I have to deal with until the next release. That, or the 2.5.4 is shipping the wrong shaka-player.compiled.js upside_down_face

Correct, those changes won't make it into the v2.5.x releases since it's a breaking change. You'll need to wait until the v2.6 release to see it. Or you can pull from master and build it yourself.

You are suggesting forking shaka player and making the change myself, correct? Or is there an external API of shaka-player's that I can use to turn it off?

I was talking about forking or just editing the source and build it manually. I was suggesting a possible solution while we work on fixing #2031.

[shaka-bot]

@alexanderturinske Does this answer all your questions? Can we close the issue?

[alexanderturinske]

Awesome, thanks. I feel like I got all the information I needed. I will close the issue.