React-native-webrtc: Cannot use a new create MediaStream to display a local or remote video track?

[ibc]

Simplified scenario:

peerconnection.onaddstream = function(event) {
  var stream = event.stream;
  var videoTrack = stream.getVideoTracks[0];

  handleVideoTrack(videoTrack);
};

function handleVideoTrack(videoTrack) {
  var newStream = new MediaStream();

  newStream.addTrack(videoTrack);
}

The above code snippet is 100% valid WebRTC API (well, a bit outdated but still valid in Chrome family).

Expected behavior

I should be able to use newStream (created locally by my app) to render the videoTrack in a RTCView.

Observerd behavior

It does not work because react-native-webrtc wrongly assumes that the only streams I'm gonna render are those generated by it (via getUserMedia or remotely generated after calling setRemoteDescription()).

Comments

Relying on the original MediaStream is a show stopper. Many modern WebRTC apps manage their own locally created MediaStream objects and use stream.addTrack() and so on.

As a hack, I think I could do the following in my app:

peerconnection.onaddstream = function(event) {
  var stream = event.stream;
  var videoTrack = stream.getVideoTracks[0];

  // HACK
  videoTrack.streamReactTag = stream.reactTag;

  handleVideoTrack(videoTrack);
};

function handleVideoTrack(videoTrack) {
  var newStream = new MediaStream(videoTrack.streamReactTag); // <--- !!!

  newStream.addTrack(videoTrack);

  // and now use `stream.toURL()` and so on to render the video track.
}

I expect this hack to work since react-native-webrtc uses the natively generated reactTag property of the stream to match both the JS stream and the native JS.

BUT: As far as I see, the stream.addTrack(track) does nothing but just fire an addtrack event:

addTrack(track: MediaStreamTrack) {
   this._tracks.push(track);
   this.dispatchEvent(new MediaStreamTrackEvent('addtrack', {track}));
}

Should I assume that such an addtrack event is listened somewhere in react-native-webrtc so the track is properly added to the stream and it could be rendered?

[dguillamot]

We also are limited by this issue. We create MediaStreams dynamically using various tracks (including tracks from a PeerConnection). This works perfectly fine on web but react-native-webrtc seems unable to deal with creating new MediaStream() objects and assigning tracks to it via addTrack..

@ibc Did you fork this project and expand the functionality of addTrack to make this work? As you said, the JS code for MediaStream.addTrack in react-native-webrtc is very shallow. I think we will need to hook into the c# side of the library as well.

[ibc]

@ibc Did you fork this project and expand the functionality of addTrack to make this work? As you said, the JS code for MediaStream.addTrack in react-native-webrtc is very shallow.

I will work on this and and do a PR eventually. Indeed many things in the API must be changed and updated (we cannot live forever with the JS API of Chrome of year 2014).

[saghul]

Please take a look at https://github.com/jitsi/react-native-webrtc/pull/55