Twilio-video.js: Chrome 85 Audio Issue

[smk1992]

[email protected]

There seems to be some kind of audio issue regarding chrome 85. When a user from chrome 85 connects with a user from chrome 84. The user from chrome 85 cannot hear the other users and the other users cannot hear him.

[james-jang]

No one else having this issue with Audio? With chrome 84 - 84 it works fine, 85-85 it works fine, but when it's 85-84 it's causing issues.

[beeeemster]

We face same issue.

[nickpell]

Also having audio issues on Chrome 85.

[swarnava]

We are also receiving multiple feedback from users that audio is buggy with Chrome 85.

[makarandp0]

Thank you for reporting this issue @smk1992, I will take a look at this today. Can you share some repro steps and room-sids that reproduced this issue?

Thanks,
Makarand

[beeeemster]

Here you go @makarandp0

RMba20a3a79e5cba75dbf791907d310f40
RMe4c4ed78c17aa810f3ad0c91cb2f2c7a

👍

[makarandp0]

Thank you @beeeemster for the room-sids, I do see that in those rooms participants did send packets on audio tracks, but did not receive any. (btw in RMba20a3a79e5cba75dbf791907d310f40 participant was on Chrome 87.0.4250 (and not Chrome 85)

Unfortunately we have not been able to get a local repro of this issue - Does this issue repro consistently ? Can you give us some details on the repro steps - Things like
1) sequence of connect,
2) were tracks published as part of connect or after words
would be useful.

If you have some sample code that makes it easy to repro, that would be great too.
Thanks,
Makarand

[makarandp0]

@beeeemster - we also noticed that you are using subscriber rules to exclude some tracks - I wonder if that has something to do with repro steps.

@nickpell @swarnava @james-jang @smk1992 - Does your repro also involve subscriber rules? If anybody has simple repro steps, please post it here. It would be very helpful as we are continue to investigate further.

Thanks,
Makarand

[nickpell]

@makarandp0 I experience the issue when I pass the audio and video tracks into connect(). The issue disappears if I publish the tracks using publishTrack() and instead pass {audio: false, video: false} to connect().

[smk1992]

@makarandp0 We don't have any subscriber rules.

Wanted to provide an update as it might help others. We were able to have the audio work when we did not pass any tracks when performing connect. Before, we would be showing user's their camera preview and an audio meter for them to test their audio. So we would be passing the localTracks for audio & video.

When exploring if this issue is also present in React Starter App. We found that it had no issues for chrome 85, so we accordingly followed the same setup and did not pass any tracks, and just passed the deviceId of the camera & microphone being used in preview as preferred devices.

[jezell]

Seeing same issue here

[makarandp0]

Update: we have identified and are testing a fix for this issue. It appears to be a bug on twilio server side. We will update this thread once the fix is released (VMS-2812)

[swarnava]

https://status.twilio.com/incidents/fvlq6hww7crv

[anna-vasilko]

All, the fix was fully rolled out tonight at 01:12am PST.
You should no longer experience audio loss in such scenario. @smk1992 @james-jang @beeeemster @nickpell @swarnava

[swarnava]

Thanks @anna-vasilko !

[nickpell]

Looks good on my end, thanks!

[anna-vasilko]

Hi all,
Here is more information about what happened.

The issue was caused by a bug in our Video Media Server which existed for a long time, but Its occurrence was really low until Chrome 85 started rolling out.
More specifically, the bug was that our Video media server would not forward audio for subscribed tracks in case when they had different extmap ids than the related published track (in RTP Header Extensions). Such condition was really rare before Chrome 85.
Google Chrome team confirmed that “Previously libwerbtc/Chrome had a more or less fixed mapping between extension IDs and extension types. Due to implementation changes this is no longer the case and can change based on how/when transceiver objects are created. It's therefore best to not rely on hardcoded mapping of IDs, this is in line with the spec"

The fix in our Video media server was to no longer rely on fixed mapping of IDs.

We are now working on a more detailed document for Reason For Outage.

We would like to apologize for the impact this incident has had on your users. We are continually working to improve the quality, resilience and robustness of our systems, as our first priority. We are applying the lessons learned from these events to implement betterments that will prevent failures of this kind from impacting our customers in the future.

Best regards,
Anna Vasilko

[kumail-raza]

@anna-vasilko
Can you guys provide more details on your finding with this issue? Actually can you refer to the ticket for google chrome where I can look it for, how and what was the fix you guys actually made in this regard thanks.

[makarandp0]

Hello @kumail-raza, Here are some details on the issue.

Chrome 85 changed the way it assigns extension IDs. You can see this change in the demo here:
https://makarandp0.github.io/chromesdpdemo/

• step 1) create a peer connection
• step 2) add audio and video track
• step 3) create offer

On chrome 85 ids assigned to extension differ depending on which track - video or audio - was added first. Notice that in the generated offer:

• Chrome 84:
  Video added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
  Audio added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
• Chrome 85
  Video added first - a=extmap:14 urn:ietf:params:rtp-hdrext:ssrc-audio-level
  Audio added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
• Firefox
  Video added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
  Audio added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
• Safari
  Video added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
  Audio added first - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level

Our media server expected the ids to match between peers. With chrome 85 they wouldn't match if one peer added video track first and the other peer added audio track first - causing it to not forward audio to the peer. The fix was to not require matching extension ids. There is no google chrome ticket for the issue as change in chrome is spec compliant.

The issue affected group rooms and where the order ids between peers did not match, that would happen most typically if one participant was on chrome 85 and it joined the room initially with video only and other peers were on older chrome or other browsers.

Thanks,
Makarand