ExoPlayer + Azure Media Services + AES (SS/DASH/HLS) all fail to play correctly.

[JohnGalt1717]

[REQUIRED] Issue description

I'm using this test video that works fine in HLS on the standard android video player: Azure Tears of Steal

I'm using Xamarin with a ViewRenderer, but I've also tried it Kotlin with the same results:

• Smooth Streaming gives the following error:

com.google.android.exoplayer2.upstream.Loader$UnexpectedLoaderException: Unexpected NullPointerException: Attempt to get length of null array at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:403) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:919) Caused by: java.lang.NullPointerException: Attempt to get length of null array at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser$ProtectionParser.getProtectionElementKeyId(SsManifestParser.java:463) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser$ProtectionParser.buildTrackEncryptionBoxes(SsManifestParser.java:454) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser$ProtectionParser.build(SsManifestParser.java:445) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser$ElementParser.parse(SsManifestParser.java:147) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser$ElementParser.parse(SsManifestParser.java:129) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser.parse(SsManifestParser.java:72) at com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifestParser.parse(SsManifestParser.java:53) at com.google.android.exoplayer2.upstream.ParsingLoadable.load(ParsingLoadable.java:158) at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:381)

• If I try HLS with "(format=m3u8-aapl)" appended to the URL per Microsoft's documentation and use hlsMediaSource instead, the video plays without audio and constant stuttering. (And it never buffers, and if you pause it appears to buffer and then hitting play and the buffer is flushed.)

• If I try DASH with "(format=mpd-time-csf)" appended to the URL per Microsoft's documentation and use DashMediaSource I get the following:

Skipping atom with length > 2147483647 (unsupported). at
com.google.android.exoplayer2.extractor.mp4.FragmentedMp4Extractor.readAtomHeader(FragmentedMp4Extractor.java:412) at com.google.android.exoplayer2.extractor.mp4.FragmentedMp4Extractor.read(FragmentedMp4Extractor.java:310) at com.google.android.exoplayer2.source.chunk.InitializationChunk.load(InitializationChunk.java:89) at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:381) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:919)

[REQUIRED] Reproduction steps

1. Create a new player with exoPlayerFactory.newSimpleInstance(context, new defaultTrackSelector()) and attach it to a playerView that is full screen on the activity.
2. Create a defaultHttpDataSourceFactory()
3. Add DefaultRequestProperties.Set("Authorization", "Bearer=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cm46bWljcm9zb2Z0OmF6dXJlOm1lZGlhc2VydmljZXM6Y29udGVudGtleWlkZW50aWZpZXIiOiI5ZGRhMGJjYy01NmZiLTQxNDMtOWQzMi0zYWI5Y2M2ZWE4MGIiLCJpc3MiOiJodHRwOi8vdGVzdGFjcy5jb20vIiwiYXVkIjoidXJuOnRlc3QiLCJleHAiOjE3MTA4MDczODl9.lJXm5hmkp5ArRIAHqVJGefW2bcTzd91iZphoKDwa6w8")

For Smooth Streaming use this: streamSource = new ssMediaSource.Factory(httpDataSourceFactory).createMediaSource(uri);
For HLS use: hlsMediaSource.Factory(httpDataSourceFactory).createMediaSource(uri);
for DASH use: dashMediaSource.Factory(httpDataSourceFactory).createMediaSource(uri);

Then execute _player.prepare(mediaSource);

_player.playWhenReady = true;

[REQUIRED] Version of ExoPlayer being used

2.9.8

Xamarin Exo Bindings

[REQUIRED] Device(s) and version(s) of Android being used

Nexus 5x Emulator (9)
Pixel 3 XL Device (10)
Pixel C Emulator (10)

[marcbaechinger]

The NullPointerException for the SmoothStreaming manifest is caused by the content of the ProtectionHeader element. The parser expects base64 encoded text.

The manifest your referring to wants AES encryption which is not supported by ExoPlayer.
There is issue #3420 which is marked as low priority enhancement. We don't have plans for supporting this though.

<Protection>
   <ProtectionHeader SystemID="B47B251A-2409-4B42-958E-08DBAE7B4EE9">
       <ContentProtection schemeIdUri="urn:mpeg:dash:sea:2012" xmlns:sea="urn:mpeg:dash:schema:sea:2012">
         <sea:SegmentEncryption schemeIdUri="urn:mpeg:dash:sea:aes128-cbc:2013" />
         <sea:KeySystem keySystemUri="urn:mpeg:dash:sea:keysys:http:2013" />
         <sea:CryptoPeriod IV="0xACDEFD6AC1116B2559D417B883839D88" keyUriTemplate="https://amssamples.keydelivery.mediaservices.windows.net/?KID=9dda0bcc-56fb-4143-9d32-3ab9cc6ea80b" />
       </ContentProtection>
   </ProtectionHeader>
 </Protection>

Besides this I was not able to construct a uri which gives me a DASH or HLS manifest. Maybe it's just me, but may I ask you to provide the uris so I can look into this?

[JohnGalt1717]

@marcbaechinger Thanks.

Here's the URLS for HLS and Dash Respectively:

https://amssamples.streaming.mediaservices.windows.net/830584f8-f0c8-4e41-968b-6538b9380aa5/TearsOfSteelTeaser.ism/manifest(format=m3u8-aapl)

https://amssamples.streaming.mediaservices.windows.net/830584f8-f0c8-4e41-968b-6538b9380aa5/TearsOfSteelTeaser.ism/manifest(format=mpd-time-csf)

HLS plays audio and video in that example but stutters a lot. In my internal code it doesn't play audio. The same link and http header plays fine in the built in videoview in Android.

Dash just fails entirely as per above.

[marcbaechinger]

For DASH, the manifest seems to request AES encryption like for SmoothStreaming. This is not supported by ExoPlayer I'm afraid. ExoPlayer supports Widevine, PlayReady and ClearKey for content protection with DASH.

For HLS the player receives a 401 when trying to fetch the license. That's the header which is added to the request.

Bearer: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cm46bWljcm9zb2Z0OmF6dXJlOm1lZGlhc2VydmljZXM6Y29udGVudGtleWlkZW50aWZpZXIiOiI5ZGRhMGJjYy01NmZiLTQxNDMtOWQzMi0zYWI5Y2M2ZWE4MGIiLCJpc3MiOiJodHRwOi8vdGVzdGFjcy5jb20vIiwiYXVkIjoidXJuOnRlc3QiLCJleHAiOjE3MTA4MDczODl9.lJXm5hmkp5ArRIAHqVJGefW2bcTzd91iZphoKDwa6w8

I see the same behaviour when using curl:

curl -H "Bearer: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cm46bWljcm9zb2Z0OmF6dXJlOm1lZGlhc2VydmljZXM6Y29udGVudGtleWlkZW50aWZpZXIiOiI5ZGRhMGJjYy01NmZiLTQxNDMtOWQzMi0zYWI5Y2M2ZWE4MGIiLCJpc3MiOiJodHRwOi8vdGVzdGFjcy5jb20vIiwiYXVkIjoidXJuOnRlc3QiLCJleHAiOjE3MTA4MDczODl9.lJXm5hmkp5ArRIAHqVJGefW2bcTzd91iZphoKDwa6w8" https://amssamples.keydelivery.mediaservices.windows.net/?KID=9dda0bcc-56fb-4143-9d32-3ab9cc6ea80b

[JohnGalt1717]

@marcbaechinger It's header: Authorization, and then value: "Bearer=" (without the quotes)

It gets everything fine and plays with HLS, it just doesn't buffer properly and stutters (and no audio in other cases)

[marcbaechinger]

Thanks and sorry for being stupid.

This plays the HLS stream without problems in the demo app. Video and audio are playing without stuttering. Do you see that behaviour on different devices?

[JohnGalt1717]

@JohnGalt1717 I've tried on 3 separate devices (2 emulators and 1 physical Pixel 3 XL). All of them stutter and refuse to buffer. (pausing causes buffering but as soon as you hit play it dumps the buffer and starts stuttering.) If you've ever seen an Instagram video that has multiple segments, think that but it's after 2 or fewer seconds. If I had to guess it's the segments getting dis-guarded that were previously buffered for some reason.

For the audio issue here's the manifest of the file I'm trying to play (It's paywalled so I can't just give you a link unfortunately.)

EXTM3U

EXT-X-VERSION:4

EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="aac_UND_1_63",DEFAULT=YES,URI="QualityLevels(63999)/Manifest(aac_UND_1_63,format=m3u8-aapl)"

EXT-X-STREAM-INF:BANDWIDTH=248454,RESOLUTION=640x360,CODECS="avc1.64001e,mp4a.40.5",AUDIO="audio"

QualityLevels(163139)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=248454,RESOLUTION=640x360,CODECS="avc1.64001e",URI="QualityLevels(163139)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=306416,RESOLUTION=960x540,CODECS="avc1.64001f,mp4a.40.5",AUDIO="audio"

QualityLevels(219854)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=306416,RESOLUTION=960x540,CODECS="avc1.64001f",URI="QualityLevels(219854)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=368408,RESOLUTION=960x540,CODECS="avc1.64001f,mp4a.40.5",AUDIO="audio"

QualityLevels(280511)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=368408,RESOLUTION=960x540,CODECS="avc1.64001f",URI="QualityLevels(280511)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=441914,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.5",AUDIO="audio"

QualityLevels(352435)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=441914,RESOLUTION=1280x720,CODECS="avc1.64001f",URI="QualityLevels(352435)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=637805,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.5",AUDIO="audio"

QualityLevels(544109)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=637805,RESOLUTION=1920x1080,CODECS="avc1.640028",URI="QualityLevels(544109)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=779004,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.5",AUDIO="audio"

QualityLevels(682269)/Manifest(video,format=m3u8-aapl)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=779004,RESOLUTION=1920x1080,CODECS="avc1.640028",URI="QualityLevels(682269)/Manifest(video,format=m3u8-aapl,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=73566,CODECS="mp4a.40.5",AUDIO="audio"

QualityLevels(63999)/Manifest(aac_UND_1_63,format=m3u8-aapl)

[marcbaechinger]

It's hard for me to understand what's happening without seeing the issue or having the content. Can you play the stream with the demo app, take a bug report and then upload here? This allows me to see what the player is doing.

You can add your stream to the media.exolist.json file like this:

      {
        "name": "#6844",
        "uri": "https://amssamples.streaming.mediaservices.windows.net/830584f8-f0c8-4e41-968b-6538b9380aa5/TearsOfSteelTeaser.ism/manifest(format=m3u8-aapl)",
        "extension": "m3u8"
      }

To add the http header you can modify the DemoApplication in a hacky way and change buildHttpDataSourceFactory :

public HttpDataSource.Factory buildHttpDataSourceFactory() {
    DefaultHttpDataSourceFactory defaultHttpDataSourceFactory = new DefaultHttpDataSourceFactory(
        userAgent);
    defaultHttpDataSourceFactory
        .getDefaultRequestProperties()
        .set("Authorization", "Bearer=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cm46bWljcm9zb2Z0OmF6dXJlOm1lZGlhc2VydmljZXM6Y29udGVudGtleWlkZW50aWZpZXIiOiI5ZGRhMGJjYy01NmZiLTQxNDMtOWQzMi0zYWI5Y2M2ZWE4MGIiLCJpc3MiOiJodHRwOi8vdGVzdGFjcy5jb20vIiwiYXVkIjoidXJuOnRlc3QiLCJleHAiOjE3MTA4MDczODl9.lJXm5hmkp5ArRIAHqVJGefW2bcTzd91iZphoKDwa6w8");
    return defaultHttpDataSourceFactory;
  }

Take a bug report and upload here or send to dev.[email protected] with the issue number #6844 as subject.

The logs created by EventLogger in the demo app should tell what is happening.

(you can alternatively do the same in your app while using EventLogger but seeing whether it's working in the demo app rules out setup problems)

[marcbaechinger]

I'm using 2.11.1 btw. It's worth trying with that version instead of 2.9.8.

[JohnGalt1717]

@marcbaechinger sorry for the delay. Looks like this might have been a fixed bug between the Xamarin Exo bindings and now for HLS. I'm trying to confirm.

[marcbaechinger]

Closing due to inactivity. Please reopen if you think it needs to be reopened.