Cordova-plugin-iosrtc: Janus and cordova-plugin-iosrtc

Created on 11 Oct 2019  路  49Comments  路  Source: cordova-rtc/cordova-plugin-iosrtc

Platform information

  • Cordova version: 8.1.2
  • cordova-ios: 4.5.5
  • Plugin version: 5.0.4 (master branch)
  • iOS version: 13.1.2
  • Xcode version: 11.1(11A1027)
  • webrtc-adapter: 7.3.0
  • janus.js 0.7.4

I'm testing the plugin with the lastest version of Janus (0.7.4) and I'm having an issue with the video (audio is working well). The scenario I'm testing is a simple EchoTest between the browser and the Janus server.

I'm using this janus.js library and it works perfectly in Chrome, Firefox, Opera, even in Safari Mobile 12.1.

However, when I compile a cordova application using cordova-plugin-iosrtc the video is not established. I see that the addTrack method is called twice (see line), but the onicecandidate event of the PeerConnection is never executed. (see line)

I've also tried the M69 branch and I've noticed the same behavior.

Maybe it's a bug of the Janus.js library, but I'm surprised it doesn't happen in browsers. Do you have any idea what might be going on?

2019-10-11 17:04:50.601839+0200 Echo410[1694:485541] PluginMediaStream#init()
2019-10-11 17:04:50.602047+0200 Echo410[1694:485541] PluginMediaStream#run()
2019-10-11 17:04:50.602420+0200 Echo410[1694:485541] iosrtcPlugin#MediaStream_setListener()
2019-10-11 17:04:50.602894+0200 Echo410[1694:485709] PluginMediaStream#setListener()
2019-10-11 17:04:50.604247+0200 Echo410[1694:485541] iosrtcPlugin#RTCPeerConnection_addStream()
2019-10-11 17:04:50.604748+0200 Echo410[1694:485541] iosrtcPlugin#MediaStream_addTrack()
2019-10-11 17:04:50.604802+0200 Echo410[1694:485709] PluginRTCPeerConnection#addStream()
2019-10-11 17:04:50.605503+0200 Echo410[1694:485729] PluginRTCPeerConnection | onnegotiationeeded
2019-10-11 17:04:50.605785+0200 Echo410[1694:485709] PluginMediaStream#addTrack()
2019-10-11 17:04:50.606279+0200 Echo410[1694:485729] PluginMediaStream | OnAddAudioTrack [label:7A2D300C-EB48-4C62-8D3D-A2B4ADBD65F5]
2019-10-11 17:04:50.607138+0200 Echo410[1694:485729] PluginMediaStreamTrack#init()
2019-10-11 17:04:50.608163+0200 Echo410[1694:485729] PluginMediaStreamTrack#run() [kind:audio, id:7A2D300C-EB48-4C62-8D3D-A2B4ADBD65F5]
2019-10-11 17:04:50.609235+0200 Echo410[1694:485709] PluginMediaStream#addTrack() | audio track added
2019-10-11 17:04:50.610099+0200 Echo410[1694:485709] PluginMediaStreamTrack#deinit()
2019-10-11 17:04:50.618223+0200 Echo410[1694:485713] PluginRTCPeerConnection#addTrack()
2019-10-11 17:05:09.487143+0200 Echo410[1694:485541] iosrtcPlugin#RTCPeerConnection_createDataChannel()
2019-10-11 17:05:09.487754+0200 Echo410[1694:485970] PluginRTCPeerConnection#createDataChannel()
2019-10-11 17:05:09.489210+0200 Echo410[1694:485970] PluginRTCDataChannel#init()
2019-10-11 17:05:09.491745+0200 Echo410[1694:485729] PluginRTCPeerConnection | onnegotiationeeded
2019-10-11 17:05:09.494360+0200 Echo410[1694:485970] PluginRTCDataChannel#run()
2019-10-11 17:05:12.767994+0200 Echo410[1694:485541] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-11 17:05:12.768295+0200 Echo410[1694:485541] PluginMediaStreamRenderer#init()
2019-10-11 17:05:12.793930+0200 Echo410[1694:485541] Metal GPU Frame Capture Enabled
2019-10-11 17:05:12.794992+0200 Echo410[1694:485541] Metal API Validation Enabled
2019-10-11 17:05:12.843862+0200 Echo410[1694:485541] PluginMediaStreamRenderer#run()
2019-10-11 17:05:12.843948+0200 Echo410[1694:485541] THREAD WARNING: ['iosrtcPlugin'] took '76.036133' ms. Plugin should use a background thread.
2019-10-11 17:05:12.844242+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-11 17:05:12.844379+0200 Echo410[1694:485541] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:64.0, elementWidth:300.0, elementHeight:150.0, videoViewWidth:300.0, videoViewHeight:150.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-11 17:05:12.844586+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-11 17:05:12.844635+0200 Echo410[1694:485541] PluginMediaStreamRenderer#render()
2019-10-11 17:05:12.845045+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamTrack_setListener()
2019-10-11 17:05:12.845197+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamRenderer_close()
2019-10-11 17:05:12.845235+0200 Echo410[1694:485541] PluginMediaStreamRenderer#close()
2019-10-11 17:05:12.845259+0200 Echo410[1694:485541] PluginMediaStreamRenderer#reset()
2019-10-11 17:05:12.845417+0200 Echo410[1694:486191] PluginMediaStreamTrack#setListener() [kind:audio, id:7A2D300C-EB48-4C62-8D3D-A2B4ADBD65F5]
2019-10-11 17:05:12.845985+0200 Echo410[1694:485541] PluginMediaStreamRenderer#deinit()
2019-10-11 17:05:12.847128+0200 Echo410[1694:485541] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-11 17:05:12.847195+0200 Echo410[1694:485541] PluginMediaStreamRenderer#init()
2019-10-11 17:05:12.851151+0200 Echo410[1694:485541] PluginMediaStreamRenderer#run()
2019-10-11 17:05:12.851417+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-11 17:05:12.851535+0200 Echo410[1694:485541] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:79.0, elementWidth:1.0, elementHeight:1.0, videoViewWidth:1.0, videoViewHeight:1.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-11 17:05:12.851780+0200 Echo410[1694:485541] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-11 17:05:12.851843+0200 Echo410[1694:485541] PluginMediaStreamRenderer#render()
MediaStream bug libwebrtc third party library support
Source
picture oscarvadillog
❤1

Most helpful comment

@menelike @hthetiot VP8 issue here https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/415

picture oscarvadillog on 24 Oct 2019
👍2

All 49 comments

Thank you for reporting the issue @oscarvadillog I'm going to look in what I can do for you.
I have worked on WebRTC unified-plan support today that might help.

I will come back to you as soon as I find something.

hthetiot picture hthetiot on 11 Oct 2019

@oscarvadillog I think found why you only had only audio and no video track.

See fix c189656a98f73bbbdb9aea18f2fe65295127116e, the issue was that when using addTrack and when providing a stream argument, the stream was not properly added to the RTCPeerConnection, resuting in create of localStreams for the RTCPeerConnection only the first time, but then was ignored for the second track.

I'm now calling addStream into addTrack (that will not add the stream if already exist) and create a new one if not provided for each track. This does fix when using the following pattern.

 var localPeerStream = new MediaStream();
    localStream.getTracks().forEach(function (track) {
      console.log('addTrack', track);
      peerConnection.addTrack(track, localPeerStream);
    });

   peerConnection.getSenders() // Now have the 2 added tracks

Note: this fix has been added to master, will be released on 5.0.5 and has been merged on m69 branches also.

hthetiot picture hthetiot on 13 Oct 2019

@hthetiot thank you for your support!
Despite the fix, I still have the issue that the onicecandidate event is never called. It seems that ICE candidates are not negotiated.

2019-10-14 01:13:01.700524+0200 Echo410[2017:929030] iosrtcPlugin#new_RTCPeerConnection()
2019-10-14 01:13:01.701007+0200 Echo410[2017:929030] PluginRTCPeerConnection#init()
2019-10-14 01:13:01.702611+0200 Echo410[2017:929030] PluginRTCPeerConnectionConfig#init()
2019-10-14 01:13:01.703049+0200 Echo410[2017:929030] PluginRTCPeerConnectionConfig#init() | adding ICE server [url:'stun:_MY_STUN_SERVER_:_PORT_', username:'', password:'******']
2019-10-14 01:13:01.703307+0200 Echo410[2017:929030] PluginRTCPeerConnectionConfig#init() | adding ICE server [url:'turn:_MY_TURN_SERVER:_PORT_', username:'trial', password:'******']
2019-10-14 01:13:01.703483+0200 Echo410[2017:929030] PluginRTCPeerConnectionConstraints#init()
2019-10-14 01:13:01.703682+0200 Echo410[2017:929030] PluginRTCPeerConnection#run()
2019-10-14 01:13:01.703771+0200 Echo410[2017:929030] PluginRTCPeerConnectionConfig#getIceServers()
2019-10-14 01:13:01.703896+0200 Echo410[2017:929030] PluginRTCPeerConnectionConstraints#getConstraints()
2019-10-14 01:13:01.712706+0200 Echo410[2017:929030] THREAD WARNING: ['iosrtcPlugin'] took '12.257080' ms. Plugin should use a background thread.
2019-10-14 01:13:06.865496+0200 Echo410[2017:929030] iosrtcPlugin#RTCPeerConnection_addStream()
2019-10-14 01:13:06.866104+0200 Echo410[2017:929195] PluginRTCPeerConnection#addStream()
2019-10-14 01:13:06.866964+0200 Echo410[2017:929219] PluginRTCPeerConnection | onnegotiationeeded
2019-10-14 01:13:06.867265+0200 Echo410[2017:929195] PluginRTCPeerConnection#addTrack()
2019-10-14 01:13:10.082803+0200 Echo410[2017:929201] PluginRTCPeerConnection#addTrack()
2019-10-14 01:13:11.426791+0200 Echo410[2017:929030] iosrtcPlugin#RTCPeerConnection_createDataChannel()
2019-10-14 01:13:11.427161+0200 Echo410[2017:929201] PluginRTCPeerConnection#createDataChannel()
2019-10-14 01:13:11.427644+0200 Echo410[2017:929201] PluginRTCDataChannel#init()
2019-10-14 01:13:11.428487+0200 Echo410[2017:929219] PluginRTCPeerConnection | onnegotiationeeded
2019-10-14 01:13:11.430451+0200 Echo410[2017:929201] PluginRTCDataChannel#run()
2019-10-14 01:13:13.523868+0200 Echo410[2017:929030] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-14 01:13:13.524228+0200 Echo410[2017:929030] PluginMediaStreamRenderer#init()
2019-10-14 01:13:13.555133+0200 Echo410[2017:929030] Metal GPU Frame Capture Enabled
2019-10-14 01:13:13.556201+0200 Echo410[2017:929030] Metal API Validation Enabled
2019-10-14 01:13:13.629212+0200 Echo410[2017:929030] PluginMediaStreamRenderer#run()
2019-10-14 01:13:13.629301+0200 Echo410[2017:929030] THREAD WARNING: ['iosrtcPlugin'] took '105.510742' ms. Plugin should use a background thread.
2019-10-14 01:13:13.629607+0200 Echo410[2017:929030] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-14 01:13:13.629764+0200 Echo410[2017:929030] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:64.0, elementWidth:300.0, elementHeight:150.0, videoViewWidth:300.0, videoViewHeight:150.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-14 01:13:13.630006+0200 Echo410[2017:929030] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-14 01:13:13.630106+0200 Echo410[2017:929030] PluginMediaStreamRenderer#render()
2019-10-14 01:13:13.630445+0200 Echo410[2017:929030] iosrtcPlugin#MediaStreamRenderer_close()
2019-10-14 01:13:13.630501+0200 Echo410[2017:929030] PluginMediaStreamRenderer#close()
2019-10-14 01:13:13.630526+0200 Echo410[2017:929030] PluginMediaStreamRenderer#reset()
2019-10-14 01:13:13.630868+0200 Echo410[2017:929030] PluginMediaStreamRenderer#deinit()
2019-10-14 01:13:13.631442+0200 Echo410[2017:929030] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-14 01:13:13.631500+0200 Echo410[2017:929030] PluginMediaStreamRenderer#init()
2019-10-14 01:13:13.635779+0200 Echo410[2017:929030] PluginMediaStreamRenderer#run()
2019-10-14 01:13:13.636009+0200 Echo410[2017:929030] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-14 01:13:13.636143+0200 Echo410[2017:929030] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:79.0, elementWidth:1.0, elementHeight:1.0, videoViewWidth:1.0, videoViewHeight:1.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-14 01:13:13.636329+0200 Echo410[2017:929030] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-14 01:13:13.636375+0200 Echo410[2017:929030] PluginMediaStreamRenderer#render()
2019-10-14 01:13:13.637085+0200 Echo410[2017:929030] [LayoutConstraints] Unable to simultaneously satisfy constraints.
oscarvadillog picture oscarvadillog on 14 Oct 2019

Ok will continue to test using Janus Echo test.

hthetiot picture hthetiot on 14 Oct 2019

@hthetiot I've disabled unified plan in janus library (setting always to false) and now ICE candidates are negotiated properly.
The audio is working now. However, the event config.pc.ontrack is never called, and therefore I never have Remote Track.

2019-10-14 10:54:56.879837+0200 Echo410[2167:1017243] iosrtcPlugin#RTCPeerConnection_createOffer()
2019-10-14 10:54:56.880205+0200 Echo410[2167:1017436] PluginRTCPeerConnection#createOffer()
2019-10-14 10:54:56.880399+0200 Echo410[2167:1017436] PluginRTCPeerConnectionConstraints#init()
2019-10-14 10:54:56.880478+0200 Echo410[2167:1017436] PluginRTCPeerConnectionConstraints#init() | [offerToReceiveAudio:true, offerToReceiveVideo:true]
2019-10-14 10:54:56.880721+0200 Echo410[2167:1017436] PluginRTCPeerConnectionConstraints#getConstraints()
2019-10-14 10:54:56.881017+0200 Echo410[2167:1017243] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-14 10:54:56.881113+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#init()
2019-10-14 10:54:56.901665+0200 Echo410[2167:1017243] Metal GPU Frame Capture Enabled
2019-10-14 10:54:56.902780+0200 Echo410[2167:1017243] Metal API Validation Enabled
2019-10-14 10:54:56.919142+0200 Echo410[2167:1017436] PluginRTCPeerConnectionConstraints#deinit()
2019-10-14 10:54:56.921835+0200 Echo410[2167:1017456] PluginRTCPeerConnection#createOffer() | success callback
2019-10-14 10:54:56.957149+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#run()
2019-10-14 10:54:56.957268+0200 Echo410[2167:1017243] THREAD WARNING: ['iosrtcPlugin'] took '76.269287' ms. Plugin should use a background thread.
2019-10-14 10:54:56.957829+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-14 10:54:56.958040+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:64.0, elementWidth:300.0, elementHeight:150.0, videoViewWidth:300.0, videoViewHeight:150.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-14 10:54:56.958342+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-14 10:54:56.958471+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#render()
2019-10-14 10:54:56.958939+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamRenderer_close()
2019-10-14 10:54:56.959017+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#close()
2019-10-14 10:54:56.959058+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#reset()
2019-10-14 10:54:56.959514+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#deinit()
2019-10-14 10:54:56.960563+0200 Echo410[2167:1017243] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-14 10:54:56.960778+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#init()
2019-10-14 10:54:56.966794+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#run()
2019-10-14 10:54:56.967058+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-14 10:54:56.967213+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:79.0, elementWidth:1.0, elementHeight:1.0, videoViewWidth:1.0, videoViewHeight:1.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-14 10:54:56.967579+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-14 10:54:56.967658+0200 Echo410[2167:1017243] PluginMediaStreamRenderer#render()
2019-10-14 10:54:56.968017+0200 Echo410[2167:1017243] Failed to bind EAGLDrawable: <CAEAGLLayer: 0x282f2bfe0> to GL_RENDERBUFFER 1
2019-10-14 10:54:56.968081+0200 Echo410[2167:1017243] Failed to make complete framebuffer object 8cd6
2019-10-14 10:54:56.970152+0200 Echo410[2167:1017243] [LayoutConstraints] Unable to simultaneously satisfy constraints.
    Probably at least one of the constraints in the following list is one you don't want. 
    Try this: 
        (1) look at each constraint and try to figure out which you don't expect; 
        (2) find the code that added the unwanted constraint or constraints and fix it. 
    (Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x280c1fbb0 h=--& v=--& UIView:0x14be774f0.minY == 79   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>",
    "<NSLayoutConstraint:0x280c12d50 UIView:0x14be774f0.top == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.top   (active)>",
    "<NSLayoutConstraint:0x280c12120 'UIViewSafeAreaLayoutGuide-top' V:|-(20)-[UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide']   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x280c12d50 UIView:0x14be774f0.top == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.top   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-14 10:54:56.971403+0200 Echo410[2167:1017243] [LayoutConstraints] Unable to simultaneously satisfy constraints.
    Probably at least one of the constraints in the following list is one you don't want. 
    Try this: 
        (1) look at each constraint and try to figure out which you don't expect; 
        (2) find the code that added the unwanted constraint or constraints and fix it. 
    (Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x280c1fe80 h=--& v=--& UIView:0x14be774f0.width == 1   (active)>",
    "<NSAutoresizingMaskLayoutConstraint:0x280c1ddb0 h=-&- v=-&- WKWebView:0x14c82c600'Ionic App'.minX == 0   (active, names: '|':UIView:0x14bd10090 )>",
    "<NSAutoresizingMaskLayoutConstraint:0x280c1e5d0 h=-&- v=-&- H:[WKWebView:0x14c82c600'Ionic App']-(0)-|   (active, names: '|':UIView:0x14bd10090 )>",
    "<NSLayoutConstraint:0x280c12df0 UIView:0x14be774f0.left == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.left   (active)>",
    "<NSLayoutConstraint:0x280c12e40 UIView:0x14be774f0.right == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.right   (active)>",
    "<NSLayoutConstraint:0x280c1d9a0 'UIView-Encapsulated-Layout-Width' UIView:0x14bd10090.width == 375   (active)>",
    "<NSLayoutConstraint:0x280c12170 'UIViewSafeAreaLayoutGuide-left' H:|-(0)-[UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'](LTR)   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>",
    "<NSLayoutConstraint:0x280c122b0 'UIViewSafeAreaLayoutGuide-right' H:[UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide']-(0)-|(LTR)   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x280c12e40 UIView:0x14be774f0.right == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.right   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-14 10:54:56.972610+0200 Echo410[2167:1017243] [LayoutConstraints] Unable to simultaneously satisfy constraints.
    Probably at least one of the constraints in the following list is one you don't want. 
    Try this: 
        (1) look at each constraint and try to figure out which you don't expect; 
        (2) find the code that added the unwanted constraint or constraints and fix it. 
    (Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x280c1fbb0 h=--& v=--& UIView:0x14be774f0.minY == 79   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>",
    "<NSAutoresizingMaskLayoutConstraint:0x280c1fa70 h=--& v=--& UIView:0x14be774f0.height == 1   (active)>",
    "<NSAutoresizingMaskLayoutConstraint:0x280c1fc50 h=-&- v=-&- WKWebView:0x14c82c600'Ionic App'.minY == 0   (active, names: '|':UIView:0x14bd10090 )>",
    "<NSAutoresizingMaskLayoutConstraint:0x280c1e350 h=-&- v=-&- V:[WKWebView:0x14c82c600'Ionic App']-(0)-|   (active, names: '|':UIView:0x14bd10090 )>",
    "<NSLayoutConstraint:0x280c12da0 UIView:0x14be774f0.bottom == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.bottom   (active)>",
    "<NSLayoutConstraint:0x280c1fde0 'UIView-Encapsulated-Layout-Height' UIView:0x14bd10090.height == 667   (active)>",
    "<NSLayoutConstraint:0x280c100f0 'UIViewSafeAreaLayoutGuide-bottom' V:[UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide']-(0)-|   (active, names: '|':WKWebView:0x14c82c600'Ionic App' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x280c12da0 UIView:0x14be774f0.bottom == UILayoutGuide:0x281674d20'UIViewSafeAreaLayoutGuide'.bottom   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-14 10:54:56.974492+0200 Echo410[2167:1017243] iosrtcPlugin#RTCPeerConnection_setLocalDescription()
2019-10-14 10:54:56.974784+0200 Echo410[2167:1017437] PluginRTCPeerConnection#setLocalDescription()
2019-10-14 10:54:56.980153+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("have-local-offer")]
2019-10-14 10:54:56.986052+0200 Echo410[2167:1017456] PluginRTCPeerConnection#setLocalDescription() | success callback
2019-10-14 10:54:56.986793+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicegatheringstatechange [iceGatheringState:Optional("gathering")]
2019-10-14 10:54:56.987072+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:1493399139 1 udp 2122260223 192.168.1.12 63339 typ host generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:56.987712+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:1493399139 2 udp 2122260222 192.168.1.12 57588 typ host generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:56.989116+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:data, sdpMLineIndex:2, candidate:candidate:1493399139 1 udp 2122260223 192.168.1.12 63559 typ host generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:56.997498+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:video, sdpMLineIndex:1, candidate:candidate:1493399139 1 udp 2122260223 192.168.1.12 53233 typ host generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:56.998261+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:video, sdpMLineIndex:1, candidate:candidate:1493399139 2 udp 2122260222 192.168.1.12 55576 typ host generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.074783+0200 Echo410[2167:1017243] iosrtcPlugin#RTCPeerConnection_setRemoteDescription()
2019-10-14 10:54:57.075036+0200 Echo410[2167:1017437] PluginRTCPeerConnection#setRemoteDescription()
2019-10-14 10:54:57.076884+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("stable")]
2019-10-14 10:54:57.456039+0200 Echo410[2167:1017456] PluginRTCPeerConnection | oniceconnectionstatechange [iceConnectionState:Optional("checking")]
2019-10-14 10:54:57.457893+0200 Echo410[2167:1017456] PluginMediaStream#init()
2019-10-14 10:54:57.458157+0200 Echo410[2167:1017456] PluginMediaStreamTrack#init()
2019-10-14 10:54:57.458267+0200 Echo410[2167:1017456] PluginMediaStreamTrack#run() [kind:audio, id:janusa0]
2019-10-14 10:54:57.458323+0200 Echo410[2167:1017456] PluginMediaStreamTrack#init()
2019-10-14 10:54:57.458376+0200 Echo410[2167:1017456] PluginMediaStreamTrack#run() [kind:video, id:janusv0]
2019-10-14 10:54:57.458488+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onaddstream [janus-0AB729AA-B664-40F3-BE04-BB35AC6E3DED]
2019-10-14 10:54:57.458687+0200 Echo410[2167:1017456] PluginMediaStream#run()
2019-10-14 10:54:57.459280+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:394300051 1 tcp 1518280447 192.168.1.12 63829 typ host tcptype passive generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.460481+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:394300051 2 tcp 1518280446 192.168.1.12 63830 typ host tcptype passive generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.461064+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:2937262007 1 udp 1686052607 212.183.218.129 63339 typ srflx raddr 192.168.1.12 rport 63339 generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.461822+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStream_setListener()
2019-10-14 10:54:57.462091+0200 Echo410[2167:1017464] PluginMediaStream#setListener()
2019-10-14 10:54:57.462612+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamTrack_setListener()
2019-10-14 10:54:57.462833+0200 Echo410[2167:1017464] PluginMediaStreamTrack#setListener() [kind:audio, id:janusa0]
2019-10-14 10:54:57.462923+0200 Echo410[2167:1017243] iosrtcPlugin#MediaStreamTrack_setListener()
2019-10-14 10:54:57.463070+0200 Echo410[2167:1017464] PluginMediaStreamTrack#setListener() [kind:video, id:janusv0]
2019-10-14 10:54:57.463805+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:2937262007 2 udp 1686052606 212.183.218.129 57588 typ srflx raddr 192.168.1.12 rport 57588 generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.464703+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:2150678982 1 udp 41885439 51.38.191.249 19842 typ relay raddr 212.183.218.129 rport 63339 generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.466547+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicecandidate [sdpMid:audio, sdpMLineIndex:0, candidate:candidate:2150678982 2 udp 41885438 51.38.191.249 19879 typ relay raddr 212.183.218.129 rport 57588 generation 0 ufrag a9DV network-id 1 network-cost 10]
2019-10-14 10:54:57.467393+0200 Echo410[2167:1017456] PluginRTCPeerConnection | onicegatheringstatechange [iceGatheringState:Optional("complete")]
2019-10-14 10:54:57.468788+0200 Echo410[2167:1017456] PluginRTCPeerConnection#setRemoteDescription() | success callback
2019-10-14 10:54:57.633939+0200 Echo410[2167:1017456] PluginRTCPeerConnection | oniceconnectionstatechange [iceConnectionState:Optional("connected")]
2019-10-14 10:54:57.634571+0200 Echo410[2167:1017456] PluginRTCPeerConnection | oniceconnectionstatechange [iceConnectionState:Optional("completed")]
2019-10-14 10:54:57.650540+0200 Echo410[2167:1017456] PluginRTCDataChannel | state changed [state:Optional("open")]
oscarvadillog picture oscarvadillog on 14 Oct 2019

@oscarvadillog Master does not support UnifiedPlan (WebRTC M54) but this branch does task/libwebrtc-update-m69-unified-plan #407 aka M69.

However, it look like webrtc-adapter is overiding addStream causing issue into iosrtc addTrack.

Can you try cordova.plugins.iosrtc.registerGlobals(); after webrtc-adapter load or before Janus.init. I'm looking into a solution.

Screen Shot 2019-10-14 at 1 42 46 AM

hthetiot picture hthetiot on 14 Oct 2019

@hthetiot I've tried to load webrtc-adapter before cordova.plugins.iosrtc.registerGlobals() and I get the same behavior.

oscarvadillog picture oscarvadillog on 14 Oct 2019

@hthetiot I think the problem is that janus.js library uses ontrack event and iosrtc (master) is using onaddstream
https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/onaddstream

oscarvadillog picture oscarvadillog on 14 Oct 2019

Below when Janus started using the ontrack event:
https://github.com/meetecho/janus-gateway/pull/942

oscarvadillog picture oscarvadillog on 14 Oct 2019

I got that fare using janus echo.

Has you may see "onremotestream" get trigger (using m69 unified-plan branch).

But it trigger "Janus stopped receiving our video"

Screen Shot 2019-10-14 at 2 13 43 PM

Screen Shot 2019-10-14 at 2 09 31 PM

Using this scripts:

/* global RTCPeerConnection */
// jshint unused:false

var server = "https://janus.conf.meetecho.com/janus";

var janus = null;
var echotest = null;
var opaqueId;

var adapterVersion = '6.4.0';
var adapterUrl = "https://webrtc.github.io/adapter/adapter-" + adapterVersion + ".js";

function loadScript(scriptUrl) {
  return new Promise(function (resolve, reject) {
    // load adapter.js
    var script = document.createElement("script");
    script.type = "text/javascript";
    script.src = scriptUrl;
    script.async = false;
    document.getElementsByTagName("head")[0].appendChild(script);
    script.onload = function () {
      console.log('loadScript.loaded', script.src);
      resolve();
    };
  })
}

function TestRTCPeerConnection() {

  return loadScript(adapterUrl).then(function () {
    return loadScript('https://janus.conf.meetecho.com/janus.js').then(function () {

        // Make sure the browser supports WebRTC
        if(!Janus.isWebrtcSupported()) {
          alert("No WebRTC support... ");
          return;
        }

        var opaqueId = "echotest-" + Janus.randomString(12);

        // Initialize the library (all console debuggers enabled)
        Janus.init({debug: "all", callback: function() {

          //Janus.unifiedPlan = false;

          // Create session
          janus = new Janus(
            {
              server: server,
              iceServers: peerConnectionConfig.iceServers,
              success: function() {
                // Attach to echo test plugin
                janus.attach(
                  {
                    plugin: "janus.plugin.echotest",
                    opaqueId: opaqueId,
                    success: function(pluginHandle) {
                      echotest = pluginHandle;
                      Janus.log("Plugin attached! (" + echotest.getPlugin() + ", id=" + echotest.getId() + ")");

                      // Negotiate WebRTC
                      var body = { "audio": true, "video": true };
                      Janus.debug("Sending message (" + JSON.stringify(body) + ")");
                      echotest.send({"message": body});

                      Janus.debug("Trying a createOffer too (audio/video sendrecv)");
                      echotest.createOffer(
                        {
                          // No media provided: by default, it's sendrecv for audio and video
                          //media: { data: false },  // Let's negotiate data channels as well
                          stream: localStream,
                          success: function(jsep) {
                            Janus.debug("Got SDP!");
                            Janus.debug(jsep);
                            echotest.send({"message": body, "jsep": jsep});
                          },
                          error: function(error) {
                            Janus.error("WebRTC error:", error);
                          }
                        });
                    },
                    error: function(error) {
                      console.error("  -- Error attaching plugin...", error);
                    },
                    consentDialog: function(on) {
                      Janus.debug("Consent dialog should be " + (on ? "on" : "off") + " now");
                    },
                    iceState: function(state) {
                      Janus.log("ICE state changed to " + state);
                    },
                    mediaState: function(medium, on) {
                      Janus.log("Janus " + (on ? "started" : "stopped") + " receiving our " + medium);
                    },
                    webrtcState: function(on) {
                      Janus.log("Janus says our WebRTC PeerConnection is " + (on ? "up" : "down") + " now");
                    },
                    slowLink: function(uplink, nacks) {
                      Janus.warn("Janus reports problems " + (uplink ? "sending" : "receiving") +
                        " packets on this PeerConnection (" + nacks + " NACKs/s " + (uplink ? "received" : "sent") + ")");
                    },
                    onmessage: function(msg, jsep) {
                      Janus.debug(" ::: Got a message :::");
                      Janus.debug(msg);
                      if(jsep !== undefined && jsep !== null) {
                        Janus.debug("Handling SDP as well...");
                        Janus.debug(jsep);
                        echotest.handleRemoteJsep({jsep: jsep});
                      }
                    },
                    onlocalstream: function(stream) {
                      Janus.debug(" ::: Got a local stream :::");
                      Janus.debug(stream);

                      localVideoEl = appContainer.querySelector('.local-video');

                      // Note: Expose for debug
                      localStream = stream;

                      // Attach local stream to video element
                      localVideoEl.srcObject = localStream;
                    },
                    onremotestream: function(stream) {
                      Janus.debug(" ::: Got a remote stream :::");
                      Janus.debug(stream);

                      peerVideoEl = appContainer.querySelector('.remote-video');

                      // Note: Expose for debug
                      peerStream = stream;

                      // Attach peer stream to video element
                      peerVideoEl.srcObject = peerStream;
                    },
                    ondataopen: function(data) {
                      Janus.log("The DataChannel is available!");
                    },
                    ondata: function(data) {
                      Janus.debug("We got data from the DataChannel! " + data);
                    },
                    oncleanup: function() {
                      Janus.log(" ::: Got a cleanup notification :::");
                    }
                  }
                );
              },
              error: function(error) {
                console.error('Janus.error', error);
              }
            }
          );
        }});
      })
    });
}
hthetiot picture hthetiot on 14 Oct 2019

@hthetiot thanks!
When I use the libwebrtc-update-m69-unified-plan branch, the onremotestream isn't triggered either.
It's stucking on this line

oscarvadillog picture oscarvadillog on 14 Oct 2019

@hthetiot I was using your Janus Echo script and the libwebrtc-update-m69-unified-plan branch.
When I run the script from a current browser works properly.

EchoJanus

However, when I try to run it on a iOS 13.1.2 device (iPhone 7), the application closes just before the onremotestream event is triggered. Did it work for you on a device?

EchoJanusMobile

oscarvadillog picture oscarvadillog on 14 Oct 2019

@hthetiot I've tried to use an old janus.js library (v.0.2.4) with task/libwebrtc-update-m69 branch and it works well for me over iOS 13. Maybe it can be a workaround until unified-plan support (6.0.0) is finished. What do you think?

oscarvadillog picture oscarvadillog on 14 Oct 2019

What a pity, only the Echo demo is working with the old library.
In the videoroom demo I can join to the room but I don't see the remote video, only local.
I think that to use an old Janus library is not a good idea. I'm gonna keep trying with the latest version of janus.js

oscarvadillog picture oscarvadillog on 14 Oct 2019

I'm not sure why the Nslog crash if you comment line 747 it goes through.
Keep me posted we going to find what wrong, this issue make iosRTC better once fix because it cover more WebRTC use cases.

Note: removed NsLog streamId causing crash on m69-unified-plan #407 branch:

hthetiot picture hthetiot on 14 Oct 2019

I'm in the process to implement proper RTCSender and shim RTCTranceiverwill do a PR tomorrow.

hthetiot picture hthetiot on 14 Oct 2019

@hthetiot I've tried to use an old janus.js library (v.0.2.4) with task/libwebrtc-update-m69 branch and it works well for me over iOS 13. Maybe it can be a workaround until unified-plan support (6.0.0) is finished. What do you think?

@oscarvadillog Please try again last unified plan branch and let me know, there is important changes in it that need to be released with m69, that will land in 6.0.x.
But i will consider your request depending if I find a way to have stable unified-plan, alternatively you may try with sdpSemantics: 'plan-b', on unified-plan branch if that fix your issue.

hthetiot picture hthetiot on 15 Oct 2019

@hthetiot Using last unified plan branch I get the following error:

error-unified

oscarvadillog picture oscarvadillog on 15 Oct 2019

Checking now. Keep you posted.

hthetiot picture hthetiot on 15 Oct 2019

Sorry About that @oscarvadillog I did some mistakes it's fixed now on master and m69 banches.
See ecb9255c0bc7ed0c90334be4a80190ca1e01b294

hthetiot picture hthetiot on 15 Oct 2019

Here is the last result on janus echo tes for me.
Screen Shot 2019-10-15 at 10 50 36 AM

hthetiot picture hthetiot on 15 Oct 2019

@hthetiot it's working the janus echo for me too.
Checking videocall...

oscarvadillog picture oscarvadillog on 15 Oct 2019

It seems that the videocall is working properly on both sides using latest janus.js, adapter 6.4.0 and task/libwebrtc-update-m69-unified-plan branch. Well done @hthetiot ! <3

videocall

Checking more demos...

oscarvadillog picture oscarvadillog on 15 Oct 2019
🚀1 🎉1

@oscarvadillog Thank you a lot for testing.
I may tag here for 6.0.1 of unified-plan and keep RTCRtpTransceiver.js SHIM/SHAM for another PR
Note: https://blog.mozilla.org/webrtc/rtcrtptransceiver-explored/

hthetiot picture hthetiot on 15 Oct 2019

Closing for now but 6.0.0 is on master not 6.0.1 with unified plan and using task/libwebrtc-update-m69-unified-plan.

It seems that the videocall is working properly on both sides using latest janus.js, adapter 6.4.0 and task/libwebrtc-update-m69-unified-plan branch.
https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/408#issuecomment-542142072

Please comment on this issue only if using UnifiedPlan or Janus MCU/SFU

hthetiot picture hthetiot on 15 Oct 2019

@hthetiot Videoroom demo is failing.

2019-10-16 00:36:33.314634+0200 VideoroomDemo[4113:1474298] PluginRTCPeerConnection#createDataChannel()
2019-10-16 00:36:33.315436+0200 VideoroomDemo[4113:1474298] PluginRTCDataChannel#init()
2019-10-16 00:36:33.315937+0200 VideoroomDemo[4113:1473711] PluginRTCPeerConnection | onnegotiationeeded
2019-10-16 00:36:33.317294+0200 VideoroomDemo[4113:1474298] PluginRTCDataChannel#run()
2019-10-16 00:36:33.317415+0200 VideoroomDemo[4113:1474298] PluginRTCPeerConnection#setRemoteDescription()
2019-10-16 00:36:33.318767+0200 VideoroomDemo[4113:1473711] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("have-remote-offer")]
2019-10-16 00:36:33.319326+0200 VideoroomDemo[4113:1473711] PluginRTCPeerConnection#setRemoteDescription() | failure callback: Error Domain=org.webrtc.RTCPeerConnection Code=-1 "Failed to set remote offer sdp: Failed to set remote video description send parameters." UserInfo={NSLocalizedDescription=Failed to set remote offer sdp: Failed to set remote video description send parameters.}
oscarvadillog picture oscarvadillog on 16 Oct 2019

But @oscarvadillog Echo demo is ok?
If video room failing can you make sure you have also TURN setup

hthetiot picture hthetiot on 16 Oct 2019

Make sure you did not have false positive on echo and displayed yourself (local vs peer stream/tracks)

hthetiot picture hthetiot on 16 Oct 2019

@hthetiot Echo and Videocall are working well, I watch the remote stream correctly.
I think it's a problem with the video codec negotiation. Why isn't VP8 codec in the offer? The room is configured to use with vp8 codec.

Below SDP offer:

v=0
o=- 8129076179245921962 2 IN IP4 1.1.1.1
s=-
t=0 0
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 1.1.1.1
a=sendonly
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
m=video 9 UDP/TLS/RTP/SAVPF 96 98
c=IN IP4 1.1.1.1
a=sendonly
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=rtpmap:96 H264/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c33
a=rtpmap:98 H264/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e033
m=application 9 DTLS/SCTP 5000
c=IN IP4 1.1.1.1
a=sendrecv

Janus log:

[Wed Oct 16 09:42:56 2019] Per the answer, the publisher is NOT going to send a video stream
oscarvadillog picture oscarvadillog on 16 Oct 2019

a=sendrecv

May be my default PeerConstrainsConfig need improvements?

But check this WebRTC options also if Janus expose them

    offerToReceiveVideo: true,
    offerToReceiveAudio: true,
    //iceTransportPolicy: 'relay',
    //sdpSemantics: 'unified-plan',
    // OR sdpSemantics: 'plan-b',
    //bundlePolicy: 'max-compat',
    //rtcpMuxPolicy: 'negotiate',

More: https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/src/PluginRTCPeerConnectionConfig.swift#L26

I'm confuse @oscarvadillog What is the difference bettween Videocall "working well," (one to one?) Videoroom demo "Videoroom demo is failing." (one to many) pls ?

Why isn't VP8 codec in the offer? The room is configured to use with vp8 codec.

I'm not sure, I got H264 in and out from FreeSwitch that I can confirm, I know m69 include also vp9 and vp8

hthetiot picture hthetiot on 16 Oct 2019

@hthetiot Echo and Videocall demos work well because are using H264, just like what happens to you from FreeSwitch. However, Janus videoroom needs to set the codecs in the configuration of each room.

If I force to use H264 it works, but I'd like to be able to use the VP8 codec whenever possible. The plugin (WebRTC M69 branch) should allow it, shouldn't it? I don't know why I don't see it in the SDP offer.

Update: Confirmed, the plugin is only working with H264. When the remote peer uses vp8, it never plays on the device. I manually manipulated the SDP to include VP8 instead of H264 but it didn't work. I watched several streams and the video elements blink :(

Below a simple test to create an SDP offer:

var adapterVersion = '6.4.0';
var adapterUrl = "https://webrtc.github.io/adapter/adapter-" + adapterVersion + ".js";
function loadScript(scriptUrl) {
  return new Promise(function (resolve, reject) {
    var script = document.createElement("script");
    script.type = "text/javascript";
    script.src = scriptUrl;
    script.async = false;
    document.getElementsByTagName("head")[0].appendChild(script);
    script.onload = function () {
      console.log('loadScript.loaded', script.src);
      resolve();
    };
  })
}
function run() {
  return loadScript(adapterUrl).then(function () {
    createOffer();
  });
}
async function createOffer() {    
  const peerConnection = window.peerConnection = new RTCPeerConnection(null);
  const offerOptions = { offerToReceiveAudio: false, offerToReceiveVideo: true };
  try {
    const offer = await peerConnection.createOffer(offerOptions);
    await peerConnection.setLocalDescription(offer);
    alert(offer.sdp);
    document.getElementById('output-offer').value = offer.sdp;
  } catch (e) {
    document.getElementById('output-offer').value = `Failed to create offer: ${e}`;
  }
}
run();

and the SDP generated (VP8 is missing 馃槩):

v=0
o=- 421674057357397935 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Bc11
a=ice-pwd:CebgQBOG1HqN9LLcMumgSA5N
a=ice-options:trickle renomination
a=fingerprint:sha-256 16:DD:77:E7:35:34:27:E5:77:A7:D6:75:53:63:C6:8A:47:4F:18:15:13:F8:AF:FF:D7:DC:D8:DB:4B:1D:C6:65
a=setup:actpass
a=mid:0
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c33
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 H264/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e033
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 red/90000
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 ulpfec/90000
oscarvadillog picture oscarvadillog on 16 Oct 2019

@saghul @ibc could you please give us your opinion here? Why do you think isn't it working with VP8? Thanks in advance!

oscarvadillog picture oscarvadillog on 16 Oct 2019

@hthetiot can you run this test and check if only h264 codec is sent in the SDP offer?
Do you have any idea why not send vp8?

Will try to review this point. Did you choose WebRTC M69 framework for some special reason?

oscarvadillog picture oscarvadillog on 18 Oct 2019

@oscarvadillog I will look into it, keep you posted

hthetiot picture hthetiot on 22 Oct 2019
❤2

I can confirm that VP8 is missing in the result of createOffer().

The test is rather simple:

const pc = new RTCPeerConnection(null);
pc.createOffer({
  offerToReceiveAudio: true,
  offerToReceiveVideo: true,
}).then((v) => console.log(v.sdp))

The sdp will only contain H264. Tested with https://github.com/cordova-rtc/cordova-plugin-iosrtc/commit/cd1a545a5bdbc6afec50713068833433dc89a1b7

Also, I don't see a strict relation to janus here, the question is if the deprecation of VP8 has been done on purpose (fingers crossed that it's not)?

menelike picture menelike on 23 Oct 2019

@menelike Yes, ofc we can close this issue (Janus problem was solved) and open a new one about the VP8 deprecation.

oscarvadillog picture oscarvadillog on 23 Oct 2019

@menelike @hthetiot VP8 issue here https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/415

oscarvadillog picture oscarvadillog on 24 Oct 2019
👍2

Thank you @oscarvadillog can you test the new WebRTC.framework build here to see if VP8 is supported #412

hthetiot picture hthetiot on 24 Oct 2019

@hthetiot is still needed this test or we go directly to https://github.com/cordova-rtc/cordova-plugin-iosrtc/pull/416 ?

oscarvadillog picture oscarvadillog on 24 Oct 2019

@oscarvadillog Yes if you can test #416, I will merge in master, I have tested it via #412 but i d'like to see if m69 master build work also, otherwise i will merge #412 and #416 today I think.

hthetiot picture hthetiot on 24 Oct 2019

Hi @hthetiot , i'm facing issues with using janus video room with the plugin

Scenario:

  • Using janus video room plugin , join one by one on 3 seperate devices, one device is an iphone with iosrtc 6.1 and 2 more on different devices
  • call cordova.plugins.iosrtc.refreshVideos() after each remote stream is added

Outcome:
Later remote videos streams override the previous videos streams

Possible Cause:
In janus video room, all remote streams have the same mediastream id value "janus", since the iosrtc plugin references to streams by the mediastream id, they get overridden. https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/1626921febcb7ee349a34eb5de58a2f5ec96cac4/js/MediaStream.js#L70

ID's are supposed to be unique, so not sure if the plugin must make changes to supposed non-unique stream id's. https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/id

akilude picture akilude on 1 Nov 2019

@akilude thank you.

I think iosRTC can mitigate that technically, but I also think that Janus should actually make unique streamId, because doing a mapping or peerID and StreamID will still be limited if one peer as twice the same streamID.

Can you create a separate issue with tht same description of your initial comment as this issue you commented on was about codec and not streamID.

hthetiot picture hthetiot on 1 Nov 2019

Done, thanks.

akilude picture akilude on 1 Nov 2019

@hthetiot @oscarvadillog
hi guys, just want to follow up here as I face same issue

I read all the discussion but it's not clear how you resolved this compatibility issue

could you please help

DaveLomber picture DaveLomber on 6 May 2020

just want to follow up here as I face same issue

@DaveLomber That not the "same" issue. I hate when people are lazy and comment on a closed issue instead of:

  1. Looking at the comments to see that this issue was actually not only not related but closed on janus side and solved (https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/408#issuecomment-542142072, https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/408#issuecomment-541608914)
  2. This simply cannot be same issue because it was closed with success and nothing changed since 6.0.1 since then and iosrtc never implemted ontrack.

Please create a DETAILED saparate issue and DO NOT comment "Same issue" on closed issue if that actually NOT THE SAME ISSUE.

hthetiot picture hthetiot on 6 May 2020

could you please help

Depend how much you pay ?

hthetiot picture hthetiot on 6 May 2020

@hthetiot apologize if I pinged you at not the best day ;-D

I just see this https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/408#issuecomment-541608914 which is the same issue I face.

I'm not a lazy guy, I re-read this thread at least 3 times, but after the above comment the discussion went in some other direction an it's not clear at all how the initial issue was resolved if neither 'ontrack' support was added at plugin sided NOR 'onaddstream' returned back on Janus js

DaveLomber picture DaveLomber on 6 May 2020

See https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/408#issuecomment-541735123

Create a dedicated issue for ontrack and I will address it.

Sorry comment on closed issue hurt my feeling.

hthetiot picture hthetiot on 6 May 2020
❤1

@DaveLomber Still not issue from you; I Did that improvement for Janus should be released via 6.0.18.

hthetiot picture hthetiot on 1 Feb 2021
Was this page helpful?
0 / 5 - 0 ratings

Related issues

yogaboy picture yogaboy  路  16Comments
nickygiorgi picture nickygiorgi  路  15Comments
smuddy picture smuddy  路  15Comments
zeijibashuai picture zeijibashuai  路  4Comments
ducile picture ducile  路  4Comments