Hls.js: Audio-only multi-language playback failure

[grabofus]

What version of Hls.js are you using?

v0.13.2

What browser and OS are you using?

Chrome on macOS

Test stream:

https://hls-js.netlify.app/demo/?src=https%3A%2F%2Ftest-9zhh43hw.s3.amazonaws.com%2Fplaylist.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsImR1bXBmTVA0IjpmYWxzZSwibGV2ZWxDYXBwaW5nIjotMSwibGltaXRNZXRyaWNzIjotMX0=

Checklist

• [x] The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• [x] The issue occurs in the stable client on https://hls-js.netlify.com/demo and not just on my page
• [x] The issue occurs in the latest client on https://hls-js-latest.netlify.com/demo and not just on my page
• [x] The stream has correct Access-Control-Allow-Origin headers (CORS)
• [x] There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Play the stream

Expected behavior

Audio stream should play with 2 available audio tracks. (English & Dubbing)
_Stream plays fine on Safari's native implementation_

Actual behavior

Stream never starts to play.

Console output

Using Hls.js config: {debug: true, enableWorker: true, liveBackBufferLength: 900}
[log] > loadSource:https://test-9zhh43hw.s3.amazonaws.com/playlist.m3u8
[debug] > Loading playlist of type manifest, level: 0, id: null
[debug] > Calling internal loader delegate for URL: https://test-9zhh43hw.s3.amazonaws.com/playlist.m3u8
[log] > trigger BUFFER_RESET
[log] > set autoLevelCapping:-1
[log] > attachMedia
[log] > media source opened
[log] > manifest loaded,1 level(s) found, first bitrate:831270
[log] > 2 bufferCodec event(s) expected
[log] > startLoad(-1)
[log] > switching to level 0
[debug] > Loading playlist of type level, level: 0, id: 0
[debug] > Calling internal loader delegate for URL: https://bitmovin-a.akamaihd.net/content/sintel/hls/audio/stereo/en/128kbit.m3u8
[log] > main stream-controller: STOPPED->IDLE
[log] > audio tracks updated
[warn] > No default audio tracks defined
[log] > Now switching to audio-track index 0
[log] > audio stream:STOPPED->IDLE
[log] > audio stream:IDLE->PAUSED
[log] > subtitle tracks updated
[log] > level 0 loaded [0,452],duration:905
[log] > Loading 0 of [0 ,452],level 0, currentTime:0.000,bufferEnd:0.000
[log] > demuxing in webworker
[log] > main stream-controller: IDLE->FRAG_LOADING
[log] > Loaded 0 of [0 ,452],level 0
[log] > Parsing 0 of [0 ,452],level 0, cc 0
[log] > main stream-controller: FRAG_LOADING->PARSING
[log] > main:discontinuity detected
[log] > main:switch detected
[log] > manifest codec:mp4a.40.2,ADTS data:type:2,sampleingIndex:3[48000Hz],channelConfig:2
[log] > parsed codec:mp4a.40.5,rate:48000,nb channel:2
[log] > audio sampling rate : 48000
[log] > main track:audio,container:audio/mp4,codecs[level/parsed]=[mp4a.40.2/mp4a.40.5]
[log] > Parsed audio,PTS:[0.000,1.963],DTS:[0.000/1.963],nb:92,dropped:0
[log] > main stream-controller: PARSING->PARSED

[grabofus]

I've just run across this issue, as it seemed similar.
https://github.com/video-dev/hls.js/issues/2330

Downgrading to v0.11.0 also fixes playback in my case.

[robwalch]

Here's what's going on:

1. The buffer-controller guesses bufferCodecEventsExpected = 2:

  onManifestParsed (data: { altAudio: boolean }) {
    // in case of alt audio 2 BUFFER_CODECS events will be triggered, one per stream controller
    // sourcebuffers will be created all at once when the expected nb of tracks will be reached
    // in case alt audio is not used, only one BUFFER_CODEC event will be fired from main stream controller
    // it will contain the expected nb of source buffers, no need to compute it
    this.bufferCodecEventsExpected = this._bufferCodecEventsTotal = data.altAudio ? 2 : 1;
    logger.log(`${this.bufferCodecEventsExpected} bufferCodec event(s) expected`);
  }

1. There is only one codec and pending track (audio) so bufferCodecEventsExpected is decremented to 1:

  onBufferCodecs (tracks: TrackSet) {
    // if source buffer(s) not created yet, appended buffer tracks in this.pendingTracks
    // if sourcebuffers already created, do nothing ...
    if (Object.keys(this.sourceBuffer).length) {
      return;
    }

    Object.keys(tracks).forEach(trackName => {
      this.pendingTracks[trackName] = tracks[trackName];
    });

    this.bufferCodecEventsExpected = Math.max(this.bufferCodecEventsExpected - 1, 0);
    if (this.mediaSource && this.mediaSource.readyState === 'open') {
      this.checkPendingTracks();
    }
  }

1. createSourceBuffers is not called because bufferCodecEventsExpected still equals 1.

  checkPendingTracks () {
    let { bufferCodecEventsExpected, pendingTracks } = this;

    // Check if we've received all of the expected bufferCodec events. When none remain, create all the sourceBuffers at once.
    // This is important because the MSE spec allows implementations to throw QuotaExceededErrors if creating new sourceBuffers after
    // data has been appended to existing ones.
    // 2 tracks is the max (one for audio, one for video). If we've reach this max go ahead and create the buffers.
    const pendingTracksCount = Object.keys(pendingTracks).length;
    if ((pendingTracksCount && !bufferCodecEventsExpected) || pendingTracksCount === 2) {
      // ok, let's create them now !
      this.createSourceBuffers(pendingTracks);
      this.pendingTracks = {};
      // append any pending segments now !
      this.doAppending();
    }
  }

[robwalch]

@grabofus the test stream is no longer available. Can you repost or provide parent manifest details and encoding details?

[grabofus]

@robwalch apologies, I've reuploaded the manifest

[robwalch]

Got it. Thanks @grabofus!

#EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="en",NAME="English"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="dubbing",NAME="Dubbing",URI="https://.../stereo/none/128kbit.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=831270,CODECS="mp4a.40.2",AUDIO="stereo"
https://.../stereo/en/128kbit.m3u8

I've also created another test stream that uses fmp4 segments rather than ts segments by removing the video tracks and matching the pattern in your stream where there is a stream inf with default audio and then alt tracks.

I have a partial fix, but unfortunately hls.js still loads the main playlist. This behavior is just engrained in how hls.js works because it always worked on the assumption that the main stream had video and the alt audio would load on and buffer on top. Doesn't make sense when all you have it audio, but that's how it currently works. For more details see #2773.