Twilio-video.js: Guidance for CPU usage?

[vincentwoo]

I have a product that uses twilio-video.js for P2P video calls, usually 1-to-1. I have gotten quite a few complaints from customers about CPU usage being high and causing problems. Does anyone have any guidance? I'm using the default getLocalTracks sans custom options, and I'm not sure if I should be doing differently, as collecting data about this is very hard.

Additionally, is there any forthcoming work regarding adaptive tweaking of the stream to hit either CPU or bandwidth thresholds? I'm given to understand that browsers lean mostly on (thirsty) CPU encoders for now, but hopefully this library can work around that when able.

[manjeshbhargav]

Hi @vincentwoo ,

unfortunately, WebRTC as a standard does not provide any way to tackle CPU issues. However, that does not mean we can't come up with some strategies. I would suggest you try out the following:

In Chrome, you can pass these non-standard CPU-based constraints to connect:

const room = await connect(token, {
  chromeSpecificConstraints: {
    mandatory: {
      // Enable CPU overuse detection
      googCpuOveruseDetection: true,
      // Not sure about this, but Google Hangouts uses it
      googCpuOveruseEncodeUsage: true,
      // The CPU % threshold for overuse
      googCpuOveruseThreshold: 85
    }
});

For all browsers in general though, I would suggest playing around with MediaTrackConstraints that deal with video capture resolution and frame rate, along with setting reasonable bandwidth limits using maxAudioBitrate and maxVideoBitrate.

Please let me know if this works for you.

Thanks,

Manjesh Malavalli
JSDK Team

[diegoarcega]

@manjeshbhargav
https://media.twiliocdn.com/sdk/js/video/releases/2.0.0/docs/global.html#ConnectOptions
There's no chromeSpecificConstraints in the connectionOptions object, is it still possible to add these configs?