Wpt: Need mock MediaStream data for some WebRTC tests

[foolip]

Currently 6 files in webrtc/ use getUserMedia(), and these will simply time out if run on a vanilla browser with no extra command line flags.

To run these WebRTC tests, a MediaStream is needed, so one of the following is needed:

• WebDriver APIs that make the getUserMedia() calls work, bypassing permissions and providing mock data.
• Depend on canvas.captureStream() or video.captureStream() (not widely supported)

See also https://github.com/w3c/web-platform-tests/issues/5671, the permissions problem would also have to be addressed to make the tests work.

[soareschen]

In many of the tests, the only reason getUserMedia() is called is because we need to pass on an instance of MediaStreamTrack. Ideally I'd like to be able to create mock instances of MediaStreamTrack for testing without having to depend on getUserMedia().

There is an alternative hackish way of creating instance of MediaStreamTrack, via new RTCPeerConnection().addTransceiver().receiver.track. But transceivers are not yet supported on most browsers, so we still use getUserMedia() for the moment.

[foolip]

@soareschen, doesn't any data need to pass through the MediaStreamTrack for the tests to work? Certainly some tests should exercise that code path.

[foolip]

@youennf @burg @eric-carlson, any thoughts on how to do this? I think we're leaning heavily towards some kind of WebDriver mocking API, see the second bullet of https://github.com/w3c/wptdashboard/issues/125.

[soareschen]

@foolip It depends on what is being tested. We haven't done much interoperability tests in WPT to make sure the media content is streamed properly to the other peer. You are right that _some_ tests still need real MediaStreamTrack to work, so having mock MediaStream is essential for testing WebRTC.

[youennf]

For WebKit, captureStream would be an option but ICE candidate filtering would probably make the test fail. Using getUserMedia seems the best option and is closer to what most web developers will do anyway.

[jan-ivar]

FWIW it should be simple to whip up fake audio tracks using web audio. See my old blog.

[foolip]

@agouaillard, can you poke this issue when you have an update on https://github.com/w3c/web-platform-tests/issues/5563#issuecomment-297281843?

[alijuma]

Is this still blocked on adding a WebDriver API?

[jugglinmike]

To run these WebRTC tests, a MediaStream is needed, so one of the following
is needed:

• WebDriver APIs that make the getUserMedia() calls work, bypassing
  permissions and providing mock data.
• Depend on canvas.captureStream() or video.captureStream() (not widely
  supported)

I think we should prefer the second approach because it will improve coverage of real-world code paths. @youennf is right to point out that using streams produced by getUserMedia is "what most web developers will do anyway," but most web developers will not be using fake streams produced by a
WebDriver-enabled stub of getUserMedia. They might be making applications where peers share pixel data in real time, though.

Browser support for canvas.captureStream is not as high as getUserMedia, but browser support for the as-yet-unspecified WebDriver endpoint is even worse :)

I've submitted a patch along these lines here: gh-10746. @jan-ivar you may be interested in that because I had to expand on the approach from your blog post in order to get things working in Chromium.

Is this still blocked on adding a WebDriver API?

@alijuma: we may get some clarity from the review of that patch.

[foolip]

@jugglinmike if we go with something like https://github.com/w3c/web-platform-tests/pull/10746, I wonder what you think we should do for testing of getUserMedia itself, i.e., the mediacapture-streams directory. @guidou, WDYT?

[jugglinmike]

I have some ideas about this, but I'll need a little more space to explain. I'll follow up with you offline before elaborating here on GitHub.

[foolip]

With https://github.com/web-platform-tests/wpt/pull/10746 this is arguably resolved, but testing of getUserMedia() itself is still a problem. I'll file another issue.