Jitsi-meet: Corrupt video from iPhone8 on App Store Jitsi Meet

[pcarlsym]

Calling from iPhone 8 to Desktop produces artefacts as seen in the attached image. It appears to be related to 720p. A few calls were successful, all having resolution lower than 720p.

Jitsi Meet version: 1.11

Relevant stream from chrome://webrtc-internals:

Statistics ssrc_151823555_recv
cname:KiqroAujb5f7gll
msid:E9FD8DE7-0F04-4C11-BFCD-B40F62CE341D 652ECF66-89F5-4C6E-AD91-78B1CBA8BE54
mslabel:E9FD8DE7-0F04-4C11-BFCD-B40F62CE341D
label:652ECF66-89F5-4C6E-AD91-78B1CBA8BE54
timestamp   06/11/2017, 11:39:59
bytesReceived   4940056
codecImplementationName libvpx
framesDecoded   2244
mediaType   video
packetsLost 4
packetsReceived 5433
qpSum   54308
ssrc    151823555
transportId Channel-audio-1
googCaptureStartNtpTimeMs   3718953523730
googCodecName   VP8
googContentType realtime
googCurrentDelayMs  189
googDecodeMs    4
googFirsSent    0
googFrameHeightReceived 1280
googFrameRateDecoded    30
googFrameRateOutput 30
googFrameRateReceived   30
googFrameWidthReceived  720
googInterframeDelayMax  37
googJitterBufferMs  174
googMaxDecodeMs 5
googMinPlayoutDelayMs   0
googNacksSent   4
googPlisSent    14
googRenderDelayMs   10
googTargetDelayMs   189
googTrackId 652ECF66-89F5-4C6E-AD91-78B1CBA8BE54

[saghul]

Hi there! Thanks for the detailed report! Can you please confirm us your Chrome version?

[pcarlsym]

Version 62.0.3202.75 (Official Build) (64-bit)

[gpolitis]

@saghul do you believe this is something related to react-native and not the bridge? @pcarlsym are you saying this is relatively reliably reproducible?

[pcarlsym]

It's very reproducible, as long as 720p is produced. And on iPhone 8 only, with iPhone 6 and 7 video is fine.

Sometimes I got 640x360 resolution and then it worked. It was quite random though, and I was not able to get it by throttling the bandwidth (just went from 720p to no video). I was also not able to enforce a lower initial resolution by modifying jitsi config, is that supposed to work on iOS?

[saghul]

I was also not able to enforce a lower initial resolution by modifying jitsi config, is that supposed to work on iOS?

Nope, iOS will always send 720p.

[saghul]

do you believe this is something related to react-native and not the bridge?

TBH, I don't know. It looks something bridge / browser related since it has started to show up only recently, and we haven't updated the WebRTC build.

[gpolitis]

@pcarlsym does this happen on multi-party calls only, or does it happen on 1-1 calls as well. We've tried to repro it internally but we were not able to. Do you think we could schedule a debugging session where you use my debugging deployment at some point?

[pcarlsym]

@gpolitis It happens for both. It seems related to VP8 and 720p. Btw. is H.264 ever used? At some point I had lots of calls phone to phone without repro. I think jitsi was using p2p and H.264 then.

We can schedule a session, preferably CET office hours.

It also works with a debug build of WebRTC.framework. But the reason for that is probably that webrtc uses VGA to encode then (verified in webrtc-internals).

[gpolitis]

@pcarlsym when it happens in a one-to-one call, have you checked whether it's in peer-to-peer mode or if the call goes through the bridge?

The reason I'm asking is because I'm trying to understand where the problem is located. If the call goes through the bridge, I'll setup my deployment to have a test call (we're in the same tz so it's going to be easy to find a convenient time for both, and thank you for being available to test). However, if the problem happens while in peer-to-peer mode then, it'd be safe to assume that something is wrong on the mobile side and we should concentrate our investigation efforts there (since no media is flowing through the bridge in peer-to-peer mode).

About H.264, we used to always enable it on all 1-1 calls (deskop-mobile, desktop-desktop, etc), but @bbaldino found out that H/W acceleration was not enabled on the Mac, for some reason (possibly a regression on the webrtc side), and performance was atrocious (at least on the desktop).

@saghul can say more about what the situation is on mobile.

[bbaldino]

just to expand on the h264 bit: until recently we preferred using h264 for p2p calls specifically so we could reap benefits of hw accel on mobile and we thought it wouldn't be an issue for the other clients to use it (due to wanting simulcast and other reasons we didn't use it for bridged calls). however recently i noticed that neither hw accel nor the assembly-optimized code was being used for h264 on mac, so we disabled it entirely by default for now. you could play with this setting and see if you're still able to repro the issue.

i'm not sure if that change coincides with when you started seeing the problem, but the PR for the change is here: https://github.com/jitsi/jitsi-meet/pull/2061

[pcarlsym]

@gpolitis It happens both in p2p mode and bridge mode I believe. At least I see the following at the end of the log for the 1-1 call:

Logger.js:125 [JitsiConference.js] <r._addRemoteTracks>:  Adding remote P2P track: RemoteTrack[875cadd5, audio, p2p: true]
Logger.js:125 [JitsiConference.js] <r._addRemoteTracks>:  Adding remote P2P track: RemoteTrack[875cadd5, video, p2p: true]
Logger.js:125 [JitsiConference.js] <r._onIceConnectionEstablished>:  Starting remote stats with p2p connection

About H.264: I have seen working calls when enabling H.264 and HW H.264 is used.

Note that the issue occurs only on iPhone 8 with 720p VP8 being sent. For example a call from iPhone 6 works fine (results in 540p):

ssrc_1005107870_recv (ssrc)
Statistics ssrc_1005107870_recv
cname:lCJG1lHHvW2V9usR
msid:7395DCD1-046D-43F0-BE64-C0E0398CE79D 25AAF198-00BF-4207-A29C-B04BD77C4CB9
mslabel:7395DCD1-046D-43F0-BE64-C0E0398CE79D
label:25AAF198-00BF-4207-A29C-B04BD77C4CB9
timestamp   06/11/2017, 22:19:21
bytesReceived   3615717
codecImplementationName libvpx
framesDecoded   868
mediaType   video
packetsLost 0
packetsReceived 3583
qpSum   29637
ssrc    1005107870
transportId Channel-audio-1
googCaptureStartNtpTimeMs   0
googCodecName   VP8
googContentType realtime
googCurrentDelayMs  36
googDecodeMs    3
googFirsSent    0
googFrameHeightReceived 960
googFrameRateDecoded    30
googFrameRateOutput 30
googFrameRateReceived   30
googFrameWidthReceived  540
googInterframeDelayMax  37
googJitterBufferMs  22
googMaxDecodeMs 4
googMinPlayoutDelayMs   0
googNacksSent   0
googPlisSent    0
googRenderDelayMs   10
googTargetDelayMs   36
googTrackId 25AAF198-00BF-4207-A29C-B04BD77C4CB9

Did you try to repro with an iPhone 8? Do you get 720p VP8 on the receiver?

[gpolitis]

@saghul @lyubomir @paweldomas what do you guys think? I was actually hoping we could blame the bridge because I know how to debug that, but it looks like this is specific to iPhone 8 and 720p VP8.

[saghul]

@gpolitis Could be. Alas I don't have access to an iPhone 8 to try and repo. Tried on my 7+ to no avail. Maybe you can check if there are any new devices there at the office?

[pcarlsym]

Nope, iOS will always send 720p.

@saghul Is there some specific reason for jitsi not respecting resolution cap config for iOS specifically?

[agglrx]

We experience the same problem already for quite some time, but with an iPad Pro (and Chrome on the other side). Everything is fine when having P2P calls, but as soon as we are routed through the bridge (either 1:1 calls or more than 2 people) the issue appears. webrtc-internals reports VP8 on both streams (recv and send).

[saghul]

Is there some specific reason for jitsi not respecting resolution cap config for iOS specifically?

It's a matter of implementing proper constraints support on react-native-webrtc. Note, however, that we might end up sending a smaller resolution depending on bandwidth availability.

[saghul]

@agglrx Thanks for chiming in!

[pcarlsym]

@agglrx Do you get H.264 for the P2P calls?

[saghul]

H.264 is now disabled for P2P calls involving a desktop browser. It will only be used for mobile to mobile calls.

[pcarlsym]

Just curios because when I force disable H.264 for P2P I see the issue in P2P regardless of receiver (chrome or mobile). I.e. 100% repro when mobile sends 720p VP8.

[saghul]

I managed to get my hands on an iPad Pro running iOS 11.0.3 and couldn't repro the issue.

[pcarlsym]

We recently reproduced it on a different iPhone8 using App Store Jitsi Meet. 1-1 call to Chrome on Mac.

[pcarlsym]

Google Hangouts does not have the problem and it also receives 720p VP8.

[saghul]

Looks like this bug might be related: https://bugs.chromium.org/p/webrtc/issues/detail?id=8028 but it's supposed to be fixed in Chrome 62...

[veprof]

Same issue here (iPhone X, Win 10 Chrome Version 63.0.3239.84 (Official Build) (64-bit))
Let me know if you need help with reproduce and additional logs.

[paweldomas]

It turns out that updating WebRTC on mobile will fix the issue (work in progress)

[paweldomas]

We're still working on that, there are problems with updating WebRTC on Android. If you need this fix and are able to build your own version your can pick iOS related commits from this branch https://github.com/jitsi/react-native-webrtc/commits/M63

[pcarlsym]

The M63 fix is on master now, closing