Cordova-plugin-iosrtc: black remote video on using twilio

[hthetiot]

Please read first!

Please use Public Google Group (mailing list) for general technical discussions and questions.

• [x] I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• [x] I have provided steps to reproduce (e.g. sample code or updated extra/renderer-and-libwebrtc-tests.js file).
• [x] I have provided third party library name and version, ios, Xcode and plugin version and adapter.js version if used.

Note: If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue.

Versions affected

• Cordova version (e.g 7.1.0): 7
• Plugin version (e.g 6.0.12): master
• iOS version (e.g 10.2): 12
• Xcode version (e.g 11.1 - 11A1027): 11A1027
• WebRTC-adapter version (e.g. 7.4.0): 7.4.0
• WebRTC Framework version (e.g. JSSip 3.1.2):

Description

Black video on remote participant unless using aee2b39b2ccebd3dc01019d7c8ebdf2bd4c90600

aee2b39b2ccebd3dc01019d7c8ebdf2bd4c90600.diff.txt

Steps to reproduce

Use sample Cordova twilio app.

Expected results

See remote participant video using master

Actual results

Black video on remote participant

[hthetiot]

@sboudouk can you try 6.0.11 just curious.

[hthetiot]

@sboudouk I would also need if you can that you test the followinf version to try to understand what changes caused the issue

• df3105cae3de0488bc67c07e8d3e18ebd068f80d
• ac93724d617f1be2b3f6b650826b9ac668f5e17f
• c04df62fe55d8597ee71d3b96373d08bbac9fd77
• cb0aefe83fca53cf333629abeac0618d718bb31a

Notice that i suspect c04df62fe55d8597ee71d3b96373d08bbac9fd77 to have broken compatilibity.

[hthetiot]

I will not release 6.0.13 until this is solved for information :)

[sboudouk]

I will give it a shot then give you feedback there. Thanks for your work.

[hthetiot]

Last 6.0.13 issue.

[matijagrcic]

One thing to mention here is that while remote works great on iOS 13 by setting the z-index(s)
videoCSS.md and https://github.com/cordova-rtc/cordova-plugin-iosrtc/commit/aee2b39b2ccebd3dc01019d7c8ebdf2bd4c90600, it doesn't work on iOS 12. The thumbnail video is shown only until the main video starts.

Do you maybe know if there is something that needed to be coded against specifically for iOS12?

Ref:
https://github.com/cordova-rtc/cordova-plugin-iosrtc/pull/98
https://github.com/cordova-rtc/cordova-plugin-iosrtc/pull/179#issuecomment-225845135

[hthetiot]

One thing to mention here is that while remote works great on iOS 13 by setting the z-index(s)
videoCSS.md and aee2b39, it doesn't work on iOS 12. The thumbnail video is shown only until the main video starts.

Do you maybe know if there is something that needed to be coded against specifically for iOS12?

Ref:

98

#179 (comment)

This as nothing to do with the issue with Twillio.
Please create dedicated issue, out if topic may lead to temporarily ban.

[matijagrcic]

Thanks for pointing that out, wanted to provide more info as it's related to the Twilio setup and remote video.

[hthetiot]

If any Twillio user can assist and update the sample to help debug iosRTC with Twillio.
https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/index-twilio.js

You can make a PR on Sample repository.

[sboudouk]

Ok then, there are my results:

Local Participant: iPhone 11 iOS 13.4.1
Remote Participant: Samsung Android 8.0.0
Twilio Room type: Group-small, and NOT p2p.

6.11:
Local Video: Local Camera successfully displayed and sent to remote. Displayed successfully on remote device.
Remote Video: No error, but can't receive, attach, or display remote video, not even a black screen.

df3105c:

Same results as 6.11.

c04df62:

Local Video: Local Camera successfully displayed and sent to remote. Displayed successfully on remote device.
Remote Video: No error, receive remote stream, but provides a blackscreen.

Here is a screenshot of console logs in participant.on('trackSubscribed') event.

First object is an instance of: RemoteTrack , and the second one is an instance of RemoteTrackPublication.

cb0aefe:

Same results as c04df62.

I hope it will be helpful.

Thanks for your work, feel free to ask precisions.

[hthetiot]

can you try this ones @sboudouk

• https://github.com/cordova-rtc/cordova-plugin-iosrtc/commit/b632a98d59e00cecabfd473370d17b7b1e2a801f
• https://github.com/cordova-rtc/cordova-plugin-iosrtc/commit/8de622ae273f7b57d536357926db25993aa9380e
• https://github.com/cordova-rtc/cordova-plugin-iosrtc/commit/852024b6fc1d24837af29f8e05a23274eca0f167

Example:

cordova plugin remove cordova-plugin-iosrtc --verbose
cordova platform remove ios --no-save
cordova platform add ios --no-save
cordova plugin add https://github.com/cordova-rtc/cordova-plugin-iosrtc#852024b6fc1d24837af29f8e05a23274eca0f167 --verbose

[sboudouk]

Local Participant: iPhone 11 iOS 13.4.1
Remote Participant: Samsung Android 8.0.0
Twilio Room type: Group-small, and NOT p2p.

b632a98:

LocalParticipant: LocalVideoTrack is displayeds successfully. Remote video track is black.
Remote Participant: RemoteParticipant track is received with success. (Audio + Video).

8de622a:

Exact same results as b632a98.

852024b:

Also exact same results as b632a98.

Interesting fact, switching to Cordova 6.1 makes also aee2b39 act like the others (remoteParticipant is a blackscreen on iPhone.) , while on Cordova 5.1.1, on first connection, everything works fine, but once remoteParticipant unpublish his video, and publish it again, it becomes black.

[akilude]

Hi,

This doesn't seem to be affecting only twilio, observing same behaviour (remote video only black) with Janus. I am able to receive remote audio.

It works fully with plugin version 6.0.7.

Here is my config that causes the issue:
_Cordova: 9
Plugin version - 6.0.12 and master (852024b6fc1d24837af29f8e05a23274eca0f167)
iOS version (e.g 13.4.1)
Xcode version - 11.5 (11E608c)
WebRTC-adapter version 7.6.4
Janus version: v0.10.3
cordova iOS 6.1.0_

[hthetiot]

Hi,

This doesn't seem to be affecting only twilio, observing same behaviour (remote video only black) with Janus. I am able to receive remote audio.

It works fully with plugin version 6.0.7.

Here is my config that causes the issue:
_Cordova: 9 Plugin version - 6.0.12 and master (852024b) iOS version (e.g 13.4.1) Xcode version - 11.5 (11E608c) WebRTC-adapter version 7.6.4 Janus version: v0.10.3 cordova iOS 6.1.0_

Thank you, we need to find what changes introduce the issue. It almost look like a he issue is in 6.0.12 but not 6.0.11.

[hthetiot]

I will investigate Monday. We need to fix that to release 6.0.13

[hthetiot]

Can you try 6.0.13-RC2 @sboudouk @akilude see changes 402baf44af7b905a01fe0c9a9be0b944f20c0909

[hthetiot]

cordova plugin remove cordova-plugin-iosrtc --verbose
cordova platform remove ios --no-save
cordova platform add ios --no-save
cordova plugin add https://github.com/cordova-rtc/cordova-plugin-iosrtc#6.0.13-RC2 --verbose

[hthetiot]

Notice that debuging with @sboudouk we found that the issue was on swyft and here was the dirty fix 6e30d459bf2732a388d8325f34b03604986fc737 compare to clean 402baf4

[hthetiot]

Last bug for 6.0.13

[hthetiot]

Master tested succesfully with twilio 2.4.0
See test file here https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/24f8755575395778b6959896f80e5eaf646d310f/www/js/index-twilio.js
https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/commit/24f8755575395778b6959896f80e5eaf646d310f