Twilio-video.js: Behavior of connect(token, { video: true }) on device that doesn't have camera

[Edvinas01]

When trying to connect Twilio video without having any camera connected to the pc, an exception is thrown and the connection is not opened.

Code to reproduce the issue:

Twilio.Video.connect(token, {
    name: 'test-room',
    audio: true,
    video: true
}).then(function (room) {
    console.log(room);
}).catch(function (error) {
    console.log(error);
});

Expected behavior:
Browser should ask the user voice and or video permissions and after accepting, the video connection should be opened.

Actual behavior:
Exception is thrown, browser does not request for audio or video permissions.

Exception:

2017-07-12 14:07:14.057Z | WARN in [createLocalTracks #1]: Call to getUserMedia failed: MediaStreamError { name: "NotFoundError", message: "The object can not be found here.", constraint: "", stack: "" }  twilio-video.js:7817:5

Software versions:

• Browser: Firefox 54.0.1 (32-bit), Google Chrome 59.0.3071.115 (64-bit)
• Operating System: Windows 10 Pro 64-bit
• twilio-video.js: 1.1.0-rc1
• Third-party libraries: Plain JavaScript

Additional notes:
If video configuration parameter is set to false, the browser asks for voice permission and the connection is opened if the user accepts.

[markandrus]

Hi @Edvinas01,

Thanks for the report. So you are trying to connect with audio and video on a device that does not support video, i.e. no camera? The connect logic is basically

1. If ConnectOptions includes tracks, proceed to step 3;
2. Otherwise, call getUserMedia with the audio and video constraints (which both default to true) and turn these into LocalTracks.
3. Connect with the LocalTracks (either tracks or the result of getUserMedia).

So the error here appears to come from the browser's own getUserMedia. If you called

navigator.mediaDevices.getUserMedia({ audio: true, video: true })

on the same device then I think you'd get the same error (no twilio-video.js in the picture).

For expected behavior, you mentioned

Browser should ask the user voice and or video permissions and after accepting, the video connection should be opened.

So would you expect that connect proceeds with only a LocalAudioTrack, no LocalVideoTrack, despite video being set to true?

[Edvinas01]

Hello, tested

navigator.mediaDevices.getUserMedia({ audio: true, video: true })

Indeed the same error appears. It seems I can handle the error and try to re-initialize the video connection with video: false.

So would you expect that connect proceeds with only a LocalAudioTrack, no LocalVideoTrack, despite video being set to true?

Yes I'd expect the library to handle the the exception and simply proceed without a video track (and the voice permissions pop-up showing up).

[ma11achy]

Hi,

I'd also like to add that I encountered the same issue while connecting without a video device to a room with a video track attached.

I found that testing with:
navigator.mediaDevices.getUserMedia({ audio: false, video: false })
allowed me to connect to the room, but not attach the video track from the room.

What I eventually ended up doing to verify I could attach the video track, was to add a video that Twilio could enumerate.

[markandrus]

@Edvinas01

Yes I'd expect the library to handle the the exception and simply proceed without a video track (and the voice permissions pop-up showing up).

I think we are hesitant to do this, because it introduces a potentially silent failure. There are already a lot of things that can go wrong with video chat (e.g., not being able to see/hear a remote participant). I'm afraid proceeding the call w/out video without a mechanism to inform the user that video was not acquired would add confusion.

[MilllerTime]

FWIW, we use navigator.mediaDevices.enumerateDevices() (after permissions are granted) to check for missing hardware and warn the user if needed. This approach gives us (or you!) full control over the situation.

I for one was happy with this being a user-land implementation, and agree that silent failures are not a great default.

[jskrzypek]

I agree that failing silently is not a great idea, but it could make this a bit easier to adopt if some option for handling of the error was wrapped into the lib.

One option might be to accept a string for the ConnectOptions.video with some value like 'optional' that tells connect() to catch the error and try again with video: false.

Alternatively, and maybe the better choice, would be to simply accept an additional property in the options like getUserMediaRetry that could instruct connect() to try again with alternatives if it catches an error thrown by getUserMedia()...

[frederikbosch]

You might want to start with tracks: []. So, like this.

Video.connect(TOKEN, {'name': room, tracks: []});

[markandrus]

I think some good alternatives have been proposed here, so I'm going to close this issue. To be clear, if you do connect with { audio: false, video: false } or, equivalently, { tracks: [] }, you may want to acquire and publish tracks later. For example,

const room = await connect(token, { tracks: [] });

let localVideoTrack;
try {
  localVideoTrack = await createLocalVideoTrack();
} catch (error) {
  console.warn('Looks like you don\'t have a camera; proceeding without');
}

if (localVideoTrack) {
  await room.localParticipant.publishTrack(localVideoTrack);
}

enumerateDevices is a nice solution, too. You'll want to pass any resulting deviceId to createLocalVideoTrack (or createLocalAudioTrack) like so:

const localVideoTrack = await createLocalVideoTrack({ deviceId });