Peerjs: Connection drops immediately after connecting

[domschiener]

Currently using PeerJS with Meteor (here is the project: https://github.com/domschiener/peered). Everything works fine locally on the same connection, but as I deploy it to my server and try to connect with a friend, the connection is very unstable and oftentimes drops immediately after connecting.

Anyone have an idea as to why that could be?

PeerJS: Socket open Success! Your Peer ID is: Ajrgmd6A3KwDLnuoB PeerJS: Creating RTCPeerConnection. PeerJS: Listening for ICE candidates. PeerJS: Listening for `negotiationneeded` PeerJS: Listening for data channel PeerJS: Listening for remote stream PeerJS: Setting remote description RTCSessionDescription {type: "offer", sdp: "v=0 ↵o=- 6918372162384193671 2 IN IP4 127.0.0.1 ↵s…id:data ↵a=sctpmap:5000 webrtc-datachannel 1024 ↵"} Received new connection ERQFHRAHkFDRMLnJf PeerJS: Set remoteDescription: OFFER for: ERQFHRAHkFDRMLnJf PeerJS: Created answer. PeerJS: Set localDescription: answer for: ERQFHRAHkFDRMLnJf PeerJS: Received ICE candidates for: ERQFHRAHkFDRMLnJf PeerJS: Added ICE candidate for: ERQFHRAHkFDRMLnJf PeerJS: Received data channel PeerJS: Data channel connection success PeerJS: DataChannel closed for: ERQFHRAHkFDRMLnJf PeerJS: Cleaning up PeerConnection to ERQFHRAHkFDRMLnJf Connection closed```

[mikkelking]

I have had similar problems. I am considering, well no, actually starting to fold the peerjs server into Meteor. Basically because both peerjs and meteor use sockets for communications, and I see little value in maintaining 2 socket connections. I can provide my source code if I am successful and you are interested - or did you abandon the project?

[abardik]

For me it was an absent sdpMid in RTCIceCandidate (negotiator.js:298), which drops a webrtc data connection between browser and nodejs imediatelly after establishment, but only if clients in different network segments or via internet (in the same local network it works without sdpMid). This is according to the official MDN spec:

https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/addIceCandidate

When an error occurs while attempting to add the ICE candidate, the Promise returned by this method is rejected, returning one of the errors below as the name attribute in the specified DOMException object passed to the rejection handler.

TypeError
The specified candidate doesn't have values for both sdpMid and sdpMLineIndex.

[negotiator.js:298]
/** Handle a candidate. */
Negotiator.handleCandidate = function(connection, ice) {
var candidate = ice.candidate;
var sdpMid = ice.sdpMid; // add sdpMid which is absent in original peerjs
var sdpMLineIndex = ice.sdpMLineIndex;
connection.pc.addIceCandidate(new RTCIceCandidate({
sdpMid: sdpMid, // add sdpMid which is absent in original peerjs
sdpMLineIndex: sdpMLineIndex,
candidate: candidate
}));
util.log('Added ICE candidate for:', connection.peer);
}

[RamyaAshika]

What is the solution for this? I'm also using peerjs @abardik @domschiener @mikkelking

[abardik]

In lib/negotiator.js add to lines as shown above with comment // add sdpMid which is absent in original peerjs

[RamyaAshika]

OOH ok fine but I'm using this library https://cdnjs.cloudflare.com/ajax/libs/peerjs/0.3.18/peer.min.js so what I need to do now? @abardik

[abardik]

Well, I forked from 0.3.13 two years ago and changed a lot. But you can fix this by yourself in new version as well. You have to modify the source code, so:

1. Grab the non-minified version from here:
   https://cdnjs.cloudflare.com/ajax/libs/peerjs/0.3.18/peer.js
2. Go to the line 722 (Negotiator.handleCandidate)
3. Add two lines shown in previous comment or change the whole function to the following:

Negotiator.handleCandidate = function(connection, ice) {
  var candidate = ice.candidate;
  var sdpMid = ice.sdpMid; // !!! THIS LINE
  var sdpMLineIndex = ice.sdpMLineIndex;
  connection.pc.addIceCandidate(
    new RTCIceCandidate({
      sdpMid: sdpMid, // !!! AND THIS LINE
      sdpMLineIndex: sdpMLineIndex,
      candidate: candidate
    })
  );
  util.log("Added ICE candidate for:", connection.peer);
};

1. In your project include your modified library instead of from CDN.

Good luck!

[RamyaAshika]

Let me try, Many many thanks

On Wed, Nov 28, 2018 at 12:31 PM Andy Bard [e-mail] wrote:

Well, I forked from 0.3.13 two years ago and changed a lot. But you can
fix this by yourself in new version as well. You have to modify the source
code, so:

1. Grab the non-minified version from here:
   https://cdnjs.cloudflare.com/ajax/libs/peerjs/0.3.18/peer.js
2. Go to the line 722 (Negotiator.handleCandidate)
3. Add two lines shown in previous comment or change the whole
   function to the following:

Negotiator.handleCandidate = function(connection, ice) {
var candidate = ice.candidate;
var sdpMid = ice.sdpMid; // !!! THIS LINE
var sdpMLineIndex = ice.sdpMLineIndex;
connection.pc.addIceCandidate(
new RTCIceCandidate({
sdpMid: sdpMid, // !!! AND THIS LINE
sdpMLineIndex: sdpMLineIndex,
candidate: candidate
})
);
util.log("Added ICE candidate for:", connection.peer);
};

1. In your project include your modified library instead of from CDN.

Good luck!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/peers/peerjs/issues/347#issuecomment-442341314, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AkxCQgtIvZDVrFLfdI-Q2l7conFJ00ldks5uzjTdgaJpZM4IkV8R
.

[RamyaAshika]

I keep on getting this error now @abardik ICE failed, add a TURN server and see about:webrtc for more details

[RamyaAshika]

TypeError: connection.pc is undefined[Learn More] this error also @abardik

[abardik]

TURN is required if your peers cannot establish a direct connection. For example, this is possible, if one of them behind the NAT. I have to use TURN often when one of the peers is mobile client (LTE). So, try to install your own TURN or find a public one (which is hard to find). Actually, did you configured ICE servers settings? Maybe the problem is just ICE (STUN), not TURN. STUN is required for any peers that not in your network. So, if you did not configure STUN (ICE) settings, your remote peers will be unavailable.

[abardik]

You can use Google's STUN:

var config = { 'iceServers': [{ 'urls': ['stun:stun.l.google.com:19302'] }] };

and then:

var peer = new Peer(id, config);

[RamyaAshika]

After googling a lot I found these things only but I don't think so this is also working.
peer = new Peer(id,{
debug: 2
config: {'iceServers': [
{ url: 'stun:stun.l.google.com:19302' }, // Pass in optional STUN and TURN server for maximum network compatibility
{ url: 'turn:numb.viagenie.ca:3478', credential: 'muazkh', username:'[email protected]' },
{ url: 'turn:numb.viagenie.ca', credential: 'muazkh', username:'[email protected]' },
{ url: 'turn:numb.viagenie.ca:3478', credential: 'peerjsdemo', username:'[email protected]' },
{ url: 'turn:192.158.29.39:3478?transport=udp', credential: 'JZEOEt2V3Qb0y27GRntt2u2PAYA=',
username:'28224511:1379330808' }
{ url: 'turn:192.158.29.39:3478?transport=tcp', credential: 'JZEOEt2V3Qb0y27GRntt2u2PAYA=', -->
username:'28224511:1379330808' }
]}
});

and yes I'm trying to connect from the Mobile(LTE) and how to install TURN server whether its a paid one? Please let me know the possible solution. I'm struggling from last week. @abardik

[abardik]

About TypeError... I need more info from your browser console (at least file and line of the error, or trace or something). Try to include the original 0.3.18 (not minified) and then just add those two lines.

[RamyaAshika]

Sure I will let you know @abardik

[abardik]

I don't know about those TURNs in your example (it seems like old test servers from peerjs developers), but STUN is good. Remove all TURNs and try again. You can try my example above with just one Google's server ('urls' is a new standard instead of old 'url' from your example).
P.S. From my experience, iPhone LTE requires TURN anyway. At least, with my mobile carrier. So, if you connect from iPhone, try to install your own TURN.

[RamyaAshika]

/** Handle a candidate. */
Negotiator.handleCandidate = function (connection, ice) {
var candidate = ice.candidate;
var sdpMid = ice.sdpMid; // !!! THIS LINE
var sdpMLineIndex = ice.sdpMLineIndex;
connection.pc.addIceCandidate(new RTCIceCandidate({ //TypeError: connection.pc is undefined
sdpMid: sdpMid, // !!! AND THIS LINE
sdpMLineIndex: sdpMLineIndex
, candidate: candidate
}));
util.log("Added ICE candidate for:", connection.peer);
};
module.exports = Negotiator;

connection.pc.addIceCandidate(new RTCIceCandidate({ //TypeError: connection.pc is undefined

This is the line I'm getting that error

and after adding that Google STUN server,this is what I got ICE failed, add a TURN server and see about:webrtc for more details
@abardik

[RamyaAshika]

and now this is my code

 var config = { 'iceServers': 
    [{ 'urls': ['stun:stun.l.google.com:19302'] 
    }] 
    };
 peer = new Peer(id, config);

[abardik]

Config is good, but the browser says you need TURN, because ICE failed. This is the same behavior I have with iPhone. If any of those TURNs from your example did not work, you should to install your own TURN or maybe try to create account here: http://numb.viagenie.ca/ (I never tried them). You have to create that account because their TURN requires an authentication, and you should put your login/password to your config (like in your example config above).

[abardik]

For TypeError. I see that error in line 650 (peer.js). But line 650 (at least in my copy of peer.js that I just downloaded) contains:
util.log("Failed to createOffer, ", err);
As you see, there is no connection.pc in this line. Maybe you use the wrong peer.js?
Anyway, try to use the original peer.js after you install TURN. When the TURN problem is gone - come back to sdpMid problem.

[RamyaAshika]

Many many thanks abardik.Let me try and I will let you know @abardik

[RamyaAshika]

After creating an account, this is how I configured in my code. Whether it's correct or not?
var config = { 'iceServers':
[{ 'urls':'stun:stun.l.google.com:19302'},
{ 'urls': 'turn:numb.viagenie.ca', credential: 'mypassword', username:'myusername' } ] }; @abardik

[kidandcat]

Just for the code modification, you can clone the repository, make the changes, and compile, you only need to launch grunt (don't need to mess with minified code). And ofc any PR is welcome.

[abardik]

By spec it should be:

{
    "iceServers": [
        { "urls": ["stun:stun.l.google.com:19302"] },
        { "urls": ["turn:numb.viagenie.ca:3478"], "username": "###", "credential": "###" }
    ]
}

[RamyaAshika]

I tried but often I'm getting the same issue if I I tried to connect 5times,in that only 3 times works. and media streams are getting strucked its not at all moving more than 2 minutes - 3minutes @abardik

[abardik]

If you successfully established LTE connection and transfered some data and it has been live for several minutes, than I believe the reason of your initial issue is gone. Now, I think you have another problem - unstable connection. It can be anything. For example, bad LTE signal or bug in your code. You should try to write as simple test as possible to exclude your bugs.
But from the other side, your serverreflexive candidate is failed on your screenshot. Without srvflx candidate you can not rich remote peer if your device does not have a public IP (it's true for many home routers and LTE devices). If you have established LTE connection, than: a) you have a public IP and your connection is established with host candidate; or b) your screenshot is outdated because it's not possible to connect to remote peer without public IP and with failed srvflx (STUN) and/or relay (TURN) candidates.
Anyway, it's hard to say something more usefull without being on your computer, sorry.

[RamyaAshika]

Can I share my code? @abardik

[abardik]

Sure.

[RamyaAshika]

Here I have attached my file as a txt. Please change it as HTML file. Sender file is the one who is creating connection and receiver is the one who is receiving the connetion.
Sender.txt
receiver.txt

and here after getting the stream in video tag,I'm drawing it into the canvas using drawImage. Please let me know and to make the connection url it will be look like
http://ipaddress:8080/Sender.html
and for receiver it should be like,
http://ipaddress:8080/Receiver.html?roomId= //In this only we will pass our id as params.For eg,roomId=xxxxxx
@abardik

`

[abardik]

Sorry, I don't have https server to upload and try your code. It's not possible to establish a non-secured (http) connection for webrtc (only localhost, but you need LTE). I suggest to start from very simple webrtc/peerjs example to avoid your own bugs. Then, when you successfully establish LTE connection in that simple example, you can start to add your own logic.

[RamyaAshika]

But it works great in Firefox and one more doubt I'm having in firefox documentation,they said webrtc won't work in safari. Whether it will work on safari or not? @abardik

[abardik]

It will. MediaConnection will work as is. But for DataConnection you have to make a little fix to send binary data (this is not required if you plan to send strings only). To do this in DataConnection.prototype._trySend find this._dc.send and replace it with the following:

if ( ios ) this._dc.send(new Uint8Array(msg));
else this._dc.send(msg);

Or you can handle this in your main code where you calling DataConnection.send(). Maybe it has been fixed already, but in v. 0.3.13 this bug exists.

[abardik]

And you have to define ios somewhere, of course.

[RamyaAshika]

Ok, I will ask this when I will work on that issue. Right now Let me try to get the connection clearly @abardik

[Bobisback]

@abardik Is there any other bug fixes needed to get peerjs to work on ios?

I am having an issue where only on iOS, if a peer connects to iOS it does not work. But if iOS connects to anything else it works fine. Any ideas?

[abardik]

Just those two: sdpMid and Uint8Array. But in my case iOS doesn't work without TURN. Maybe it's because of my LTE provider. Wi-Fi is ok without TURN, as I remember. Didn't try on Android with LTE. Maybe it's the same.

[Bobisback]

Well I was just planning to only send strings, and my application will be wifi only. So technically it should work then?

Maybe I am just missing something. In my case it says it is connected and I get the connection event being fired off of the peer object that gives a DataConnection object, but then any interaction I have with the DataConnection object, does not work. No console logs, no errors, just nothing at all. Not sure what is going on. Super annoying though.

[Bobisback]

Can you confirm that you got peerjs to work correctly on iOS 11 in a MKWebView application? I am guessing that Meteor is some sort of cross platform web based framework like cordova?

There is tons of things saying that iOS 11 is not supported for webRTC, based on my research this is not entirely true, and in fact it works fine but you cannot get the data stream through the browser. Since we can get the data stream natively I am trying to find out if WebRTC and peerJS actually do work in iOS and Android minus the data stream issue in iOS 11. Any thoughts?

[abardik]

Yes, it works totally correctly on iOS, starting from iOS 9 or 10 when WebRTC has been announced by Apple (pure Safari and Phonegap app with WKWebView and iosrtc plugin, because WKWebView for Cordova still doesn't support WebRTC). No issues with DataConnection and MediaConnection. It's hard to deal with iosrtc plugin, but it works finally. Android the same - everything works, starting from Android 5.

[abardik]

Meteor is not lika Cordova. Meteor it's just a full-stack web framework. Cordova is a mobile framework. Cordova gives you an access to device's features. Meteor just gives you an exellent way to deal with data and UI.

[Bobisback]

Ok so you never actually created a native mobile app then? Meteor just allows you to code a website for mobile, but it is using prue safari not WKWebView. Based no ym research WKWebView is the issue.

[Bobisback]

@abardik So how do you define iOS or not in the Uint8Array bug?

Also does this refer to the person you are sending it to is on iOS or does it mean that client who is sending it is on iOS?

[abardik]

Something like this:

var isCordovaIOS = /Apple/.test(navigator.vendor) && /YourAppUA/.test(navigator.userAgent);
var isSafari = /Apple/.test(navigator.vendor) && /Safari/.test(navigator.userAgent) && ! isCordovaIOS;

[abardik]

Phonegap (Cordova) is a native app. You just use JS to code instead of Objective C or Java. Meteor is a web framework. You can not build a native app with meteor only. You have to wrap meteor app into cordova app. Then you have WKWebView in your cordova app, in which meteor web app is running. But from the WebRTC point of view it doesn't meter what web framework you're using. You have cordova implementation of WKWebView, which is on iOS has no WebRTC yet. So, you have to add iosrtc plugin in your cordova app. For Android app WebRTC just works, as well as Safari on your iPhone. But for iOS cordova app you need iosrtc plugin.

[abardik]

And yes, there are no issues with WebRTC and peerjs on all of those platforms. Only restrictions I have with WebRTC are in IE and Edge. Chrome, Firefox, Safari, UIWebView, WKWebView - all of them just work.

[Bobisback]

@abardik Cool that is good to here. Thanks for the input. At this point I just need to figure out how to piece all this together.

Right now I am doing a test with data connection and i am having issues with it in both safari and iOS MKWebView. Maybe you can help me out with it?

When setting breakpoints it would appear the issue is related to RTCPeerConnection and the event onicecandidate. With full logging, getting no errors, this only happens when another device tries to connect to safari or iOS. So the device connection works correctly and even fires the open connection event, but when i send stuff nothing happens. On iOS and Safari for some reason the onicecandidate never fires which seems to be causing the issues.

I have implemented both of your fixes and still having issues. Any ideas?

[abardik]

Uint8Array is for already established connection and just for binary data transfer. For strings only you don't need it. And, actually, I see now, that I've commented sdpMid fix almost a year ago. It works now without that fix for me. Sorry for that. Comment it and try again. Your problem is ICE. If you have no onicecandidate, you have no peer to connect to. I had the same issue on iOS on LTE without TURN. But, I think, this is just my provider's LTE network configuration. If your provider provides a direct IP for LTE, you don't need TURN. If both of your peers are in the same wi-fi network, you even dont' need STUN, just host candidates. But i'm not sure WebRTC will work on iOS without STUN config (if I remember, it didn't work for me without google STUN in peerjs config, even in the same wi-fi). So, for the first, make sure you've configured STUN (see comments above) and connect both peers to the same wi-fi network. Then look at the 'host' and 'srvflx' candidates in webrtc logs. Then configure TURN, connect through LTE and look for 'relay' candidates. 'srvflx' and/or 'relay' are required for LTE connections.

[RamyaAshika]

I have tried with COTURN configuration with my local system using my local IP address. It worked. But now, I'm trying to configure this using my public IP. Is it possible? and I'm doing this using Cygwin, can I able to configure this in my system with my public IP address? @abardik

[abardik]

Theoretically, yes. But I never configured TURN locally, so, practically, I don't know.

[Bobisback]

Hey @abardik,

Thanks a ton for the great input on this subject, you are a huge help. Currently I am only sending strings, so guessing the fix does not matter. Also I have tried with and without the stun servers, I basically copied the above suggestions to the letter. There is no change on behavior for my app. The requirement for this app is it only needs to work over wifi, nothing else. So I do not have to worry about turn servers.

You mention if you have no onicecandidate then you have no peer, but here is the log of the bnroken one.

```[Log] Connected to server as ax8p10z501g00000 (main.js, line 2166)
[Log] PeerJS: – "Socket open" (vendor.js, line 88282)
[Log] GET "http://192.168.102.146:8080/api/connected-users/" (main.js, line 2166)
succeeded in 33 ms, with body: ["ax8p10z501g00000 (You)"]
[Log] PeerJS: – "Creating RTCPeerConnection." (vendor.js, line 88282)
[Log] PeerJS: – "Listening for ICE candidates." (vendor.js, line 88282)
[Log] PeerJS: – "Listening for data channel" (vendor.js, line 88282)
[Log] PeerJS: – "Listening for remote stream" (vendor.js, line 88282)
[Log] PeerJS: – "Setting remote description" – RTCSessionDescription {type: "offer", sdp: "v=0
↵o=- 817605983474189040 2 IN IP4 127.0.0.1
↵s=…id:data
↵a=sctpmap:5000 webrtc-datachannel 1024
↵", toJSON: function} (vendor.js, line 88282)
RTCSessionDescription {type: "offer", sdp: "v=0
↵o=- 817605983474189040 2 IN IP4 127.0.0.1
↵s=…id:data
↵a=sctpmap:5000 webrtc-datachannel 1024
↵", toJSON: function}RTCSessionDescription
[Log] Got connection event in page: fmri5l4xjz000000 (main.js, line 2166)
[Log] Data connection opened. (main.js, line 2166)
[Log] Data connection setup with fmri5l4xjz000000 (main.js, line 2166)
[Log] PeerJS: – "Added ICE candidate for:" – "fmri5l4xjz000000" (vendor.js, line 88282)
[Log] PeerJS: – "Set remoteDescription:" – "OFFER" – "for:" – "fmri5l4xjz000000" (vendor.js, line 88282)
[Log] PeerJS: – "Added ICE candidate for:" – "fmri5l4xjz000000" (vendor.js, line 88282)
[Log] PeerJS: – "Created answer." (vendor.js, line 88282)
[Log] PeerJS: – "Set localDescription: answer" – "for:" – "fmri5l4xjz000000" (vendor.js, line 88282)
[Log] PeerJS: – "Added ICE candidate for:" – "fmri5l4xjz000000" (vendor.js, line 88282)

Now here is the log for the one that works.

```PeerJS:  Socket open
logging.service.ts:11 Connected to server as 7tvukmruwhq00000
logging.service.ts:11 GET "http://192.168.102.146:8080/api/connected-users/"
             succeeded in 29 ms, with body: ["ax8p10z501g00000","tvjfqg34iuo00000","7tvukmruwhq00000 (You)"]
util.js:65 PeerJS:  Creating RTCPeerConnection.
util.js:65 PeerJS:  Listening for ICE candidates.
util.js:65 PeerJS:  Listening for data channel
util.js:65 PeerJS:  Listening for remote stream
util.js:65 PeerJS:  Setting remote description RTCSessionDescription {type: "offer", sdp: "v=0
↵o=mozilla...THIS_IS_SDPARTA-63.0.3 1691102812…a=sctp-port:5000
↵a=max-message-size:1073741823
↵"}
logging.service.ts:11 Got connection event in page: tvjfqg34iuo00000
logging.service.ts:11 Data connection opened.
logging.service.ts:11 Data connection setup with tvjfqg34iuo00000
3util.js:65 PeerJS:  Added ICE candidate for: tvjfqg34iuo00000
util.js:65 PeerJS:  Set remoteDescription: OFFER for: tvjfqg34iuo00000
util.js:65 PeerJS:  Created answer.
util.js:65 PeerJS:  Set localDescription: answer for: tvjfqg34iuo00000
2util.js:65 PeerJS:  Received ICE candidates for: tvjfqg34iuo00000
util.js:65 PeerJS:  Received data channel
util.js:65 PeerJS:  Data channel connection success
logging.service.ts:11 Data connection open according to peerjs.

As you can see, it is getting the handshake, just after it creates the offer nothing happens, but on the one that works it gets a Received ICE candidates message. Also sometimes if I wait long enough I get a this on the client that is broken. The client that is connecting to the broken ones appears the same as a normal one, aka it connects and a open event shows.

[Log] PeerJS: – "iceConnectionState is disconnected, closing connections to fmri5l4xjz000000" (vendor.js, line 88282) [Log] PeerJS: – "iceConnectionState is disconnected, closing connections to fmri5l4xjz000000" (vendor.js, line 88282) [Log] Error on Receive listening: There was an error Error: Negotiation of connection to fmri5l4xjz000000 failed. (pages-dashboard-dashboard-module.js, line 238)
Any thoughts?

Thanks,
Bob

[abardik]

I have to see your candidates to say something. If you have just one candidate 127.0.0.1 (localhost), no other peer can connect to you, except others on the same device. Go to Negotiator.handleCandidate and change the last line to this:
util.log('Added ICE candidate for:', connection.peer, candidate);

[Bobisback]

This is what it outputs when it is not working in safari. Aka this browser is receiving the call.

"Added ICE candidate for:" – "3b5aflyiazv00000" – "candidate:1737421303 1 udp 1686052607 216.10.85.7 52681 typ srflx raddr 192.168.102.146 rport 52681 generation 0 ufrag ZuhX network-id  1 network-cost 10" (vendor.js, line 88282)

Here is the same safari browser when it is working, aka it is making the call.

"Added ICE candidate for:" – "3b5aflyiazv00000" – "candidate:239386166 1 udp 2122260223 192.168.102.146 62277 typ host generation 0 ufrag cr/a network-id 1 network-cost 10"

What is really interesting is the fact that When Safari is calling the other browser this line of code never runs, and it still receives the ice candidate correctly. Maybe the problem is that the peer making the connection to Safari is not adding a ice server when it should be and that is what is killing safari, it is waiting for something that is never added?

When I connect a chrome browser to a firefox one here is the output on firefox.

PeerJS:  Added ICE candidate for: 3b5aflyiazv00000 candidate:1737421303 1 udp 1686052607 216.10.85.7 50742 typ srflx raddr 192.168.102.146 rport 50742 generation 0 ufrag Vl9c network-id 1 network-cost 10
PeerJS:  Added ICE candidate for: 3b5aflyiazv00000 candidate:1086800582 1 tcp 1518280447 192.168.102.146 9 typ host tcptype active generation 0 ufrag Vl9c network-id 1 network-cost 10

And the output on chrome.

PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:0 1 UDP 2122252543 192.168.102.146 57094 typ host
util.js:65 PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:2 1 TCP 2105524479 192.168.102.146 9 typ host tcptype active
util.js:65 PeerJS:  Data channel connection success
logging.service.ts:11 Data connection open according to peerjs.
util.js:65 PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:1 1 UDP 1686052863 216.10.85.7 57094 typ srflx raddr 192.168.102.146 rport 57094

So for some reason on safari is just not passing the ice candidate correctly. On fire fox I just got a new error that said ice failed add a turn server. So I still need one even though it is local ip's?

How hard are they to setup? I have a nodejs server atm, can I set one up on that?

Any thoughts?

Edit: Interesting, if I call firefox from safari I consistently get a negotiation of connection failed, and firefox gets the add a turn server message. I guess the next step is to setup a turn server?? I was trying to avoid that :(

[abardik]

@Bobisback, You don't have to setup your own TURN for testing purposes (look at the comments above about numb.viagenie.ca).

Maybe it can be usefull for somebody, so I'll try to simplify some webrtc details here.

The webrtc idea is to connect two devices directly to each other the shortest and fastest way. If devices on the same host, the first candidate (127.0.0.1 typ host) is used. This candidate is available anytime and anywhere if your device's network adapter is not disabled. And any clients on this one host can connect to each other directly. If your devices in the same network, another candidate will come in help (local_network_address typ host). Webrtc clients use this type of candidates next after localhost. And it works because all of your local network devices have a 'physical' connections between each other. In other words, if you can ping other computer, you can reach it with webrtc.

Next, if your devices in different networks or network segments, including internet, they can't ping each other (and connect to each other with webrtc or anything else) if they don't have a direct public IP address (dynamic or static, it doesn't matter). The examples are everywhere: any office computer, most of home computers behind Wi-Fi router, etc. In this situation, STUN with UDP hole punching technique is coming to help. To be pretty simple, your device makes a request to STUN and receives a response with global IP:port, with which it connected to STUN (this IP:port is your internet provider's NAT IP:port which is used to transfer your requests and receive responses from web servers). Today, by statistics, in ~80% situations regular NATs will receive packets and transfer it to your device not only from hosts you requested, but from any other hosts. So, you can send this IP:port to another webrtc client, and that client can connect to this IP:port and send you any data. Once again, this is an external well-reached from anywhere IP:port of your provider's NAT (not yours), which is used then to transfer incoming packets to your host only. This is called a UDP hole punching (you can google a detailed explanation with examples). And this IP:port from STUN is a 'srvflx' candidate in your webrtc logs.

But in some situations (~20%) NAT accepts packets on that IP:port only from the same host you sent request to, or uses new port for every single request-response. So, if you send a request to google STUN (and NAT assign some port to you for this request), it will accept packets (or even just one response) on this port from google STUN only and from nobody else. This kind of NATs are called 'symmetrical NATs'. In this case, your webrtc client will try to use 'srvflx' candidate and will fail. And here we have TURN.

TURN provides public IP:port for you to send it to other webrtc clients as your 'relay' candidate. Webrtc clients will try to use 'relay' candidates only if they failed to connect with 'host' and 'srvflx' candidates. Actually, the algorithm a little bit smarter and uses candidates based on not only 'typ', but on 'network-cost' and other parameters. TURN works like your company's NAT and provides a real always reachable public IP:port (dynamic or static), but without 'symmetrical issue'. But all of your trafic is going through your TURN now. So, you have to place your TURN in well accessable locations, such as AWS infrastructure.

The last thing about establishing webrtc connection is a 'trickle ICE'. This technique is invented to reduce a connection time. Without it webrtc client sends SDP only when it got all candidates from its network, STUN and TURN servers. This collection work could take a lot of time (seconds). So, with trickle ICE enabled, webrtc client sends SDP as soon as it obtained its first candidate (which is localhost typ host). Another peer can reach this candidate if they both works on the same host. So, it makes sense to send an SDP immediately. Then, local network candidates are obtained (it's very fast) and sent, and it's possible to connect to all computers in your local network (no delay, practically). If your peers in diffefrent networks, the 'srvflx' candidate should be obtained from STUN (1-2 seconds) and used to try to connect. And, finally, TURN candidates (another 1-2 seconds). So, to avoid this delay, webrtc clients try to connect as soon as possible and then they send additional candidates, and another peers try to connect with those new candidates until they are connected. Even if a connection is already established, webrtc clients can use another candidates if they decided (based on network-cost parameter, for example) that new candidate is better then old one. Sometimes this technique fails, and we have dropped connections in a few seconds after successful connect.

So, if all of your devices are in the same network segment (wi-fi or ethernet), you don't have to use any STUN or TURN, because all of your devices are 'pingable' to each other. But, if you have some kind of personal or corporate firewall or something that can block incoming or even outgoing trafic on your devices, you could hit this (fire)wall. I never experienced this kind of problems before, but theoretically it could be.

Try to find any pure webrtc example (no peerjs or other libraries). If you have no netwrok limitations you will connect in any direction. And then you can continue your experiments with peerjs.

[abardik]

I just tried all of my peerjs clients (windows chrome, windows electron, mac electron, windows nodejs, android chrome, android phonegap, ios safari, ios phonegap) with and without TURN/STUN. All of them work in any direction when in local Wi-Fi network with zero ICE configuration (iceServers: []). Then, I tried LTE on iOS, and it works fine without TURN and even without STUN. So, my carrier provides public IP addresses for LTE clients, cool. Then, I tried Wi-Fi hot spot on iOS device and connected Android client to LTE through iOS wi-fi hot spot. In this configuration Android didn't work without TURN. So, Wi-Fi hot spot acts like a symmetrical NAT or proxy. But in Wi-Fi network all clients just work without any TURN or even STUN, because all of them can reach each other directly.

[Bobisback]

Thanks for all the great information. Going to dissect this later today or tomorrow. Based on my knowledge and and this information, my guess is that there is a NAT or some network configuration on this wifi that is breaking safari and iOS. I will do more testing as soon as I get a chance.

[RamyaAshika]

Hi @abardik

I have tried to connect with one peer to another peer. getusermedia works on one peer and I'm sharing the stream using MediaStream API.
//Making call
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({
audio: false
, video: true
}).then(function (mediaStream) {
conn = peer.call(destId, mediaStream);
myvideoplay.srcObject = mediaStream;
myvideoplay.addEventListener('loadeddata', function () {
myvideoplay.play(); // start playing
update(); //Start rendering
});
}, function (err) {
alert(err);
});
}

//Receiving call
peer.on('call', function (call) {
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({
audio: false
, video: true
}).then(function (mediaStream) {
call.answer(mediaStream);
call.on('stream', function (mediaStream) {
myvideo1.srcObject = mediaStream;
myvideo1.addEventListener('loadeddata', function () {
myvideo1.play(); // start playing
update(); //Start rendering
});
});
}, function () {
alert("Error! Make sure to click allow when asked for permission by the browser");
});
}
else {
console.log('browser wont support')
}
});

but its returning as a black screen in ios. Please Help needed.

[Bobisback]

@RamyaAshika Ya I am not sure if it related to what you are doing, but in my case ios 10+ has 100% confirmed to break navigator.mediaDevices.getUserMedia access on embedded web browsers. With no plans to support it in the future. You can google and find lots of information on the subject. it works in safari but not on embedded iOS devices aka MKWebView. Based on my research i have found some workarounds. The one I am trying now is through AudioInput and AudioNode. More specifically this link right here was the magic for me: https://github.com/edimuj/cordova-plugin-audioinput/issues/64

Aka safari mobile navigator.mediaDevices.getUserMedia works, but if you use an embedded webview aka MKWebView what cordova and basically every other hybrid mobile app is built in navigator.mediaDevices.getUserMedia just flat does not work.

[abardik]

@RamyaAshika, Are you sure your

var video = document.querySelector('#your_video_id');
video.setAttribute('autoplay', 'autoplay');
video.setAttribute('playsinline', 'playsinline');
video.setAttribute('webkit-playsinline', 'webkit-playsinline');

But if it doesn't help, just bring back the

[abardik]

@Bobisback, I don't remember exactly on what platform mediaDevices.getUserMedia doesn't work (maybe, as you said, it's cordova webview), but old style navigator.getUserMedia works very well. So, I use this:

window.getUserMedia = function(constraints, onstream, onerror) {
    if ( navigator.mediaDevices ) {
        navigator.mediaDevices.getUserMedia(constraints).then(onstream).catch(onerror);
    }
    else {
        var _getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
        _getUserMedia(constraints, onstream, onerror);
    }
}

Works on all platforms (ios cordova webview, ios safari, mac safari, android cordova webview, android chrome, windows chrome, windows firefox, windows electron, mac electron).

[abardik]

@RamyaAshika, And for iOS phonegap/cordova app you have to add this in your config.xml to allow video to play automatically without user action:

<platform name="ios">
    <preference name="MediaPlaybackRequiresUserAction" value="false"/>
    <preference name="mediaTypesRequiringUserActionForPlayback" value="0"/>
    <preference name="AllowInlineMediaPlayback" value="true"/>
</platform>

[RamyaAshika]

Okay fine I ll try and let you know @abardik and one more thing in my IOS mobile its showing error message like "The current browser does not support webRTC" like that its showing. Please whether it will work for all version ?? @abardik

[Bobisback]

@Bobisback, I don't remember exactly on what platform mediaDevices.getUserMedia doesn't work (maybe, as you said, it's cordova webview), but old style navigator.getUserMedia works very well. So, I use this:

window.getUserMedia = function(constraints, onstream, onerror) {
  if ( navigator.mediaDevices ) {
      navigator.mediaDevices.getUserMedia(constraints).then(onstream).catch(onerror);
  }
  else {
      var _getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
      _getUserMedia(constraints, onstream, onerror);
  }
}

Works on all platforms (ios cordova webview, ios safari, mac safari, android cordova webview, android chrome, windows chrome, windows firefox, windows electron, mac electron).

Based on my testing and research, on old versions of iOS this worked. On iOS 10+ both of these failed to work. Though I was not aware of the autoplay for iOS so that might have been the issue for me. I will test with it. When I tested it do matter what version of getUserMedia I used it was null on iOS cordova. So I do not believe it is related to the autoplay you speak off.

Thanks,
Bob

[abardik]

@RamyaAshika, WebRTC in Safari is supported started from iOS 9, I beleive. If you use any other browser, like Chrome or Firefox, I don't know do they support WebRTC or not, but Safari does.

[abardik]

@Bobisback, Yes, of course, WebRTC is absent in any cordova webview. I proceeded from the fact that you've already installed iosrtc plugin, as I mentioned earlier:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc

and initialized it:

document.addEventListener('deviceready', function () {
  if ( window.device.platform === 'iOS' && cordova.plugins.iosrtc ) {
    cordova.plugins.iosrtc.registerGlobals();
  }
}

Then, you can use all of the following:

navigator.getUserMedia
navigator.webkitGetUserMedia
navigator.mediaDevices.getUserMedia
navigator.mediaDevices.enumerateDevices
window.RTCPeerConnection
window.webkitRTCPeerConnection
window.RTCSessionDescription
window.RTCIceCandidate
window.MediaStream
window.webkitMediaStream
window.MediaStreamTrack

And be sure you read this:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/blob/master/docs/iosrtc.md

and this:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/blob/master/docs/videoCSS.md

[RamyaAshika]

Many many thanks I 'll try and let you know @Bobis But I tried in one of my
iPhone it's thrown a alert like it's not supported by webRTC I donno why it
happened and as well as autoplay is not working.

On Thu, 13 Dec 2018 10:55 pm Andy Bard, [e-mail] wrote:

@RamyaAshika https://github.com/RamyaAshika, WebRTC in Safari is
supported started from iOS 9, I beleive. If you use any other browser, like
Chrome or Firefox, I don't know do they support WebRTC or not, but Safari
does.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/peers/peerjs/issues/347#issuecomment-447051779, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AkxCQtfIdLoRoKaWIeULrEcx9tITMRK8ks5u4o2GgaJpZM4IkV8R
.

[RamyaAshika]

On IOS10 it's not working @abardik Its showing error message like WEBRTC is not supported by the browser like that its showing.

[RamyaAshika]

@Bobisback IOS 12 it is working but on IOS 10 it's not at all working what to do. At least I need to make this from IOS 9 @abardik.Please help needed.

[abardik]

@RamyaAshika, yes, sorry, it's included to Safari in iOS 11. Before that you can use webrtc in native apps only. For example, you can build cordova app and include iosrtc plugin to support WebRTC, but not in Safari.

[RamyaAshika]

Oops !! Whether it's a different procedure or else the same steps I need to
do? Because I'm building ionic 3 angular 5 app. From my app only I'm
opening this link in external browser's. Can I able to move further by
using this?

Thanks
RAMYA D

On Fri, 14 Dec 2018 10:30 pm Andy Bard, [e-mail] wrote:

@RamyaAshika https://github.com/RamyaAshika, yes, sorry, it's included
to Safari in iOS 11. Before that you can use webrtc in native apps only.
For example, you can build cordova app and include iosrtc plugin to support
WebRTC, but not in Safari.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/peers/peerjs/issues/347#issuecomment-447386801, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AkxCQqRKlycXzPYkecpa6uguCXTRew_qks5u49k7gaJpZM4IkV8R
.

[abardik]

@RamyaAshika, As I know, Ionic doesn't provide its own webrtc support. You have to add iosrtc plugin to your app (see my comments above). Then you can use WebRTC API with some restrictions, which you can find in iosrtc plugin docs.

[RamyaAshika]

Okay I 'll try and let you know. Thanks @Andy Bard

On Fri, 14 Dec 2018 11:24 pm Andy Bard, [e-mail] wrote:

@RamyaAshika https://github.com/RamyaAshika, As I know, Ionic doesn't
provide its own webrtc support. You have to add iosrtc plugin to your app
(see my comments above). Then you can use WebRTC API with some
restrictions, which you can find in iosrtc plugin docs.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/peers/peerjs/issues/347#issuecomment-447402070, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AkxCQo-sGa-ayUvogQi-DmmzWdScmsPoks5u4-XDgaJpZM4IkV8R
.

[Bobisback]

@abardik So based on the information posted here, if you want to use webRTC with cordova, it is not possible to use Peerjs?

If you use iosrtc then peerjs cannot work correctly right? So the options are to roll my own signalling using iosrtc or not supporting ios?

I am just confused about how iosrtc can work along side peerjs, guessing they cannot work correctly together? Or if I have both libraries installed iosrtc just magically overrides the functions that peerjs use? Just a bit confused how this fits togeth.

I am also trying to build an ionic project using cordova. Again I got it working with peerjs, but similar to the data connection if I call to the ios device, I get a voice connection one way but not the other, if I call from the ios device it seems to work fine with audio going both ways?

I am getting the stream through audio input, so I know 100% that the audio stream is there. Also I do not need video so, just need audio.

Any thoughts?

P.S where does the below code go?

document.addEventListener('deviceready', function () {
  if ( window.device.platform === 'iOS' && cordova.plugins.iosrtc ) {
    cordova.plugins.iosrtc.registerGlobals();
  }
}

I am guessing in the device ready equivalent for ionic. If so then my next question is how do I import it correctly.

[abardik]

@Bobisback, peerjs works perfectly together with iosrtc. iosrtc plugin includes libwebrtc library to your cordova app and implements WebRTC API, which is absent in cordova webview. Peerjs uses WebRTC API. Without iosrtc.registerGlobals you have to write different code for browsers and for cordova app. With registersGlobals all WebRTC API provided by iosrtc plugin is registered globally (and it's the same as WebRTC API in any browser), so peerjs can use it like in any browser, and your code is the same for all platforms. Please, read iosrtc and peerjs docs.

[abardik]

I don't know the details of ionic framework, but if I understand right, it runs inside cordova app. registerGlobals should be called in cordova's deviceready event handler (you can place this code anywhere in your JS; it will be executed after cordova is initialized and all plugins are loaded).

[RamyaAshika]

Ok fine I want to pass the stream to peer connection without enabling their camera. In one html
peer.on('connection', function (conn1) {
conn1.on('data', function (data) {
var conn = peer.connect(data);
conn.send("from desktop");
conn.on('open', function (id) {
const sUsrAg = navigator.userAgent;
//making call to pass annotations stream to receiver
var captureStream = document.getElementById('main-canvas').captureStream();
conn = peer.call(data, captureStream);
});
});
});

This is how I'm passing the stream to the other user. In another html
peer.on('call', function (call) {
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({
audio: false
, video: true
}).then(function (mediaStream) {
call.answer(mediaStream);
}, function (err) {
//alert("Error! Make sure to click allow when asked for permission by the browser");
alert(err);
});
}
else {
console.log('browser wont support')
}
});

In this I don't want to enable their camera,I just want to get the remote stream from one html. Is that possible? @abardik @Bobisback

[Bobisback]

@abardik Thanks for the comments, I am still trying to get iosrtc installed atm. Getting some errors related to swift and I cannot seem to figure out the issues with it. Once I get this sorted I will move back to trying to get peerjs to work.

If you have any ideas here is the thread: https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/issues/338

@RamyaAshika If you are trying to just send a stream from one peer to another with nothing going the other direction, peerjs docs says this is possible. Here is the documentation on the subject.

When calling or answering a call, a MediaStream should be provided. The MediaStream represents the local video (webcam) or audio stream and can be obtained with some (browser-specific) version of navigator.getUserMedia . When answering a call, the MediaStream is optional and if none is provided then a one-way call is established. Once the call is established, its open property is set to true.

I cannot comment on your code, since I decided not to use the get media devices function. I only needed audio so I decided to go with using audioinput plugin to get the audio stream.

[RamyaAshika]

ooh, k What I'm really trying to do is,

I had made this successfully on chrome, firefox and so on by making two media connections I can able to share the video as well as on top of that I can able to pass canvas as a media stream(my drawings on top of the video)but now it's not working in safari.

When we making two media connections using getUserMedia(). In safari disables the first connection and show the black screen instead of apply on top. I want to pass my canvas stream from peer.call() without getUserMedia.Please help needed. Is that possible? Because getUserMedia is the one asking permission for the camera.

peer.on('call', function (call) {
call.answer(stream); //results as ReferenceError: stream is not defined
}

because without getUserMedia it shows like this but what I want to do is I want to pass only video streams to my video tag @Bobisback

[Bobisback]

@RamyaAshika I am not sure what your "canvas stream" is. When you answer the call you can pass in null or you need to pass in a mediastream object. I am still not 100% sure what you are trying to do. If you are just trying to get the video stream without audio, then I do not see why you cannot just pass in the options saying no audio. Seems like the getUserMedia should be fine for you to use?

@abardik So I managed to get iosRTC working, I think, not sure how to tell tbh... ahaha. I am still having issues though. I seem to be having an issue with playback on iOS though. Based on the logs I am getting the streams correctly, and the connections are working, but no audio is coming through on iOS.

Also if I connect two apple devices together, it does the same thing as the data connection. It acts like it is connecting but nothing happens. Does not appear to get the streams either.

[abardik]

@RamyaAshika, ReferenceError: stream is not defined - it's just because you have no stream variable defined, not a media stream itself. Check your variables names (in your previous example it was called mediaStream, not stream). And about video-only streams. Of course, you can use them as @Bobisback mentioned above, calling getUserMedia with { video: true, audio: false }.

[abardik]

@Bobisback I never had any problem with iosrtc (or any other plugin) installation because I use Phonegap Build to build my projects, and every well-developed plugin is compatible with Phonegap Build. So, just to make things simple I refused to learn and use xcode, native SDKs, etc. But that's great if you figured it out on xcode. About your problem. Did you try a simple webrtc example without peerjs? You have to get a pure webrtc working first. Then you can proceed to peerjs with a 100% guarantee that your network configuration is not an issue.

[abardik]

@Bobisback One time I had a problem with Chrome-Safari webrtc streams because of different media codecs available in these two browsers. Safari has H.264, Chrome has VP8 and H.264, and it uses VP8 by default. When you establish a direct p2p webrtc connection both browsers can negotiate about codecs, and everything works well. But in my case it was intermediate SIP server (freeswitch), to which both browsers were connected to. All traffic was going through this server mixed together and converted to a single codec. The configuration of that server allows to specify what codec to convert to, and if you choose VP8, Safari shows black screen instead of video. The same with audio codecs. They are different on different browsers. But if browsers are connecting directly (as peerjs allows us to do) they can negotiate what codecs to use. The same thing with cordova app. I had green (not black) screen until I enabled both codecs on freeswitch. So, there are many reasons why you can get a black screen. Try to start from a simplest webrtc example to confirm that it's a peerjs issue, not a network's or browser's one.

[Bobisback]

@abardik Thanks for the input, I will try to get a simple webrtc application going next. I am only trying to send audio, I do not care about video at all.

Do you happen to have a simple sample project for phonegap that uses just WebRTC, or maybe some guides?

I need to figure out quickly if webrtc is a good solution, if I cannot get this to work on iOS then I need to move on to greener pastures.

Just need to figure what to install. The basics of webrtc is just opening a connection with each other. I will look at the docs and see if I can figure it out.

[abardik]

@Bobisback https://github.com/muaz-khan/WebRTC-Experiment
You will find a lot of examples over there. WebRTC is a good solution, I assure you. It works fine on every OS and almost every browser.

[RamyaAshika]

Actually this is a bug in safari "New getUserMedia() request kills existing stream track" https://webrtchacks.com/guide-to-safari-webrtc/ but I want to get the stream fron peer.call without getUserMedia. Is it possible? or else I want to pass the stream from one html to another html. Any idea,Please help needed @abardik

[abardik]

@RamyaAshika I can confirm that bug. I workaround it with a global audio stream captured when the call is started and switching video streams from front and back camera (in the following getUserMedia calls with video only constraints) and adding/removing them to/from PeerConnection with addTrack/removeTrack. It works fine. But yes, if you try to getUserMedia a media source (for example, microphone) that you already captured in previous getUserMedia call and didn't release it yet - you'll receive an error that media source is already captured.
I'm not sure, if you can use MediaStream from canvas in cordova app (because iosrtc plugin has its own non-standard and non-totally-compatible implementation of MediaStream). But I think you can do this in Safari, because it has a pure standard MediaStream. So, yes you can try to grab MediaStream from one canvas, add it to PeerConnection, and then attach it to another canvas on the remote peer. I never tried canvas streams, but theoretically it sounds good.

[Bobisback]

@abardik Sorry I know I am bugging you a ton, but I really wanna get this working. I was looking through the samples for the webRTC. Right now my objective is to get a working connection between iOS and Android devices using peer to peer audio, at first I wanna get it working in browsers. That's all I need. I am very unfamiliar with WebRTC, it even took me a second to grasp PeerJs and compared to WebRTC Peerjs seems super simple. Could you point me in the right direction on what sample I should focus on to get basic peer to peer audio going, ideally without a server. I am just not sure what all is needed for WebRTC to work, ideally I want the simplest solution, if that means manually typing ips in that is fine. If I can get away from using an server that is good as well.

Once I get it working in the browser, then I will move on to using it in cordova. The cool part is ionic is built on top of phonegap so it should still work in ionic. There is a lot of pieces to this pizzule and my brain is exploding trying to figure out how to fit them all together. Now trying to learn WebRTC is making it more complicated. Any pointers on where to start would be great.

As far as getting iosrtc working I had to modify the xcode project as iosrtc does not work with the latest version of xcode. I am going to try and downgrade my version I am using to see if it makes it easier.

Thanks,
Bob

[abardik]

@Bobisback,

https://www.html5rocks.com/en/tutorials/webrtc/basics/

I think, you can start from this very short but powerfull tutorial with live examples inside. You will start from getUserMedia and AudioContext, then go to local peer-to-peer video chat (in one browser tab, but you will learn a lot about internals of webrtc), and finally you'll try their fully functional video chat with their own signaling server (analogue of peerjs server).

To be sure that webrtc is good enough for your project you can just load https://appr.tc in any browser, including iOS Safari and Android Chrome, and you will see that everything just works.

After that you can proceed with peerjs. peerjs is not only client wrapper for webrtc API. There is a server side of peerjs, which does exactly what appr.tc server does - transfers SDP and candidates between peers (via WebSockets). Without signaling server your users should exchange SDP and candidates manually (for example, via skype, because SDP is just a text). But, of course, it's not an option. So, in your final app you have to implement not only client GUI, but install and maintain your own copy of peerjs server (you can do it on your own windows or linux machine inside your corporate network or you can rent a minimal AWS virtual machine with nodejs installed).

Anyway, I suggest to start from that tutorial. You will learn enough to proceed next by yourself.

Good luck!

[Bobisback]

Thanks for the link, I will check it out for sure. I found another really good tutorial here as well. It is not as complciated as I imagined. Just trying to get the signalling separated from the actual webRTC stuff. I am not actually 100% on what part is signalling and what is not, and what part is client and what part is server. Still going through the tutorials so should have it figured out shortly.

Based on what I have read and seen (besides iosrtc to make ios work) I do not need to actually install anything to use the WebRTC stuff, aka there are no npm install or package.json things I need to add? It just works out of the box with no imports or anything? If that is the case I just need to figure out how to use it with typescript, guessing just define a var somewhere and use like normal.

As far as https://appr.tc/ goes, that is really cool, but I am not sure it applies to me. How can I go to that on an iOS or android device in cordova? That is really what I am trying to test is cordova, not the browsers. The issue is iOS safari works but cordova does not use iOS safari it uses MKWebView or the other one SFSafariViewController or something and those two do not work with WebRTC, my understanding is that iosrtc fixes this. So is there a way to setup iosrtc and a basic cordova app and just direct link the https://appr.tc/ website in the code? Would this even work? Guessing not.

So my goal right now is to create a basic cordova project with iosrtc and WebRTC and get some basic communication between iOS devices.

One other issue I run into a lot is when I get an audio stream from a remote host in android, I get a error related to origin not accepted. Any ideas what this is?

[abardik]

@Bobisback Signaling is a transfer of SDP (offer and answer) and candidates between peers. It should be done with a help of signaling server (which is peerjs server is) or manually by users (for example, via any IM). WebRTC can not transfer SDP and candidates by itself. WebRTC API is only this: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API

Yes, you do not have to install anything if you will use a plain WebRTC API and your users will exchange their SDPs and candidates manually. But if you want to simplify your development process and your users' lives, you have to use some existing library to abstract from WebRTC API and the most important - from signaling part. That's why peerjs is needed. With peerjs you can forget about candidates, SDP and other internal things of WebRTC. You just create data and media connections, and peerjs will do the rest, including a creation and a transfer of SDP and candidates.

Regarding npm install, package.json, etc. For client side you don't have to install anything. You can just include <script src="peer.js"> in your html. For server side you should install nodejs, npm and run your own peerjs server.

There is no cordova webview that has WebRTC support. You can bring WebRTC to cordova webview with the help of iosrtc plugin as discussed earlier.

Yes, you can redirect your cordova app directly to any URL. This is simpler than you think. Just create a Phonegap Build app (https://build.phonegap.com) and set <content src="https://appr.tc"/> in your config.xml. Of course, you should add iosrtc and other plugins to your config.xml (for example, plugins to ask user permissions to use camera and microphone, whitelist plugin to access different URLs, and others). But it's another story. But yes, you can do it, and it's the simplest way to build what you want to build, as I understood.

All 'origin' issues (if we are speaking about cordova) can and should be solved with whitelist plugin (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-whitelist/), which allows you to allow your cordova app to access some (or all) remote resources.

[Bobisback]

So I created a new project to get this working. It would seem that iosrtc is not compatible with the newest changes to the webRTC library?

example, the old function was called RTCPeerConnection.addStream, new one is called RTCPeerConnection.addTrack.

With my current code base on ios I am getting errors related to this.

TypeError: _this.pc.addTrack is not a function. (In '_this.pc.addTrack(track, _this.localStream)', '_this.pc.addTrack' is undefined)

Any ideas how to get a around this?

[abardik]

@Bobisback That's right. Add your MediaTrack to MediaStream and use addStream for ios cordova app. You can do this on all your clients, because the old API is still supported.

[abardik]

@Bobisback Even if the old API will be totally removed from all browsers, you still can use it in your cordova project, because you can choose what versions of plugins to use. The current versions of WKWebView (1.1.4) and iosrts (4.0.2) both support addStream. For the rest of clients I suggest to use addTrack or to write a shim.

[Bobisback]

@abardik Thanks for the input. So I got WebRTC working now in a test project with ionic 4. So this time if I do browser to device it works fine. i get audio transfer on both with no issues. Connecting looks fine in the logs. it appears to work both ways. But if I connect iOS to iOS, all the logs look great. But I just get no audio playing. I have added the auto play stuff to the info.plist, and since the audio works fine on the browser connected to iOS, this means at least I know I am getting mic and the speakers work. I am not sure what is going on. Any ideas why iOS to iOS would just have no audio, should I start looking at codex's I guess?

Frist iOS Device log

[Log] Starting to get stream audioinput (cordova.js, line 1732)
[Log] Got stream audioinput (cordova.js, line 1732)
[Log] Message from client: Asking to join room Local_Channel (cordova.js, line 1732)
[Log] Adding local stream. (cordova.js, line 1732)
[Log] Received request to create or join room Local_Channel (cordova.js, line 1732)
[Log] Room Local_Channel now has 2 client(s) (cordova.js, line 1732)
[Log] Client ID CwNYTbbYTkSVg2fHAAAA joined room Local_Channel (cordova.js, line 1732)
[Log] Client Id: CwNYTbbYTkSVg2fHAAAA Joined Local_Channel (cordova.js, line 1732)
[Log] Client said: got user media (cordova.js, line 1732)
[Log] Client received message: – {sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"}Object
[Log] >>>>>>> maybeStart()  – false (2) (cordova.js, line 1732)
MediaStream {id: "1e85cd93-bedf-48b4-9ee7-023db6591fdc", active: true, onaddtrack: null, onremovetrack: null, getAudioTracks: function, …}
true
[Log] >>>>>> creating peer connection (cordova.js, line 1732)
[Log] Created RTCPeerConnnection (cordova.js, line 1732)
[Log] isInitiator – false (cordova.js, line 1732)
[Log] Sending answer to peer. (cordova.js, line 1732)
[Log] Remote stream added. (cordova.js, line 1732)
[Log] setLocalAndSendMessage sending message – {sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"}Objectsdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s=-
↵t=0 0
↵a=group:BUNDLE audio
↵a=msid-semantic: WM…"type: "answer"Object Prototype__defineGetter__(propertyName, getterFunction)__defineSetter__(propertyName, setterFunction)__lookupGetter__(propertyName)__lookupSetter__(propertyName)constructor: function()hasOwnProperty(propertyName)isPrototypeOf(property)propertyIsEnumerable(propertyName)toLocaleString()toString()valueOf()
[Log] Client said: [object Object] (cordova.js, line 1732)
[Log] icecandidate event:  – RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, …} (cordova.js, line 1732)
RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, type: "icecandidate", target: RTCPeerConnection, …}RTCPeerConnectionIceEvent
[Log] End of candidates. (cordova.js, line 1732)

Second ios device log

[Log] Starting to get stream audioinput (cordova.js, line 1732)
[Log] Got stream audioinput (cordova.js, line 1732)
[Log] Message from client: Asking to join room Local_Channel (cordova.js, line 1732)
[Log] Adding local stream. (cordova.js, line 1732)
[Log] Received request to create or join room Local_Channel (cordova.js, line 1732)
[Log] Room Local_Channel now has 1 client(s) (cordova.js, line 1732)
[Log] Client ID N4sWw5nhUaC0UN37AAAJ created room Local_Channel (cordova.js, line 1732)
[Log] Client Id: N4sWw5nhUaC0UN37AAAJ Created Local_Channel (cordova.js, line 1732)
[Log] Client said: got user media (cordova.js, line 1732)
[Log] Another peer made a request to join room Local_Channel (cordova.js, line 1732)
[Log] This peer is the initiator of room Local_Channel! (cordova.js, line 1732)
[Log] Client received message: – "got user media" (cordova.js, line 1732)
[Log] >>>>>>> maybeStart()  – false (2) (cordova.js, line 1732)
MediaStream {id: "2800710b-ba0b-440d-b451-21f8274be567", active: true, onaddtrack: null, onremovetrack: null, getAudioTracks: function, …}
true
[Log] >>>>>> creating peer connection (cordova.js, line 1732)
[Log] Created RTCPeerConnnection (cordova.js, line 1732)
[Log] isInitiator – true (cordova.js, line 1732)
[Log] Sending offer to peer (cordova.js, line 1732)
[Log] setLocalAndSendMessage sending message – {sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"}Object
[Log] Client said: [object Object] (cordova.js, line 1732)
[Log] Client received message: – {sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"}Object
[Log] Remote stream added. (cordova.js, line 1732)
[Log] icecandidate event:  – RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, …} (cordova.js, line 1732)
RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, type: "icecandidate", target: RTCPeerConnection, …}RTCPeerConnectionIceEvent
[Log] End of candidates. (cordova.js, line 1732)

[Bobisback]

Do I need to manually install adapter.js to my cordova project? if so how do you do that?

maybe that is what I am missing.