Twilio-video.js: Ways to debug/improve CPU usage & performance?

[gregoryjjb]

Our twilio-video application has very high CPU utilization. Things we've tried to improve it:

• Limit the resolution of videos to 1920p
• Use H264 and PCMU as the default video/audio encoding
• Cap audio/video bitrate at 16k and 64k respectively
• Set the secret Google flags like so (stolen from an answer to another similar issue here):

chromeSpecificConstraints: {
  mandatory: {
    // Enable CPU overuse detection
    googCpuOveruseDetection: true,
    // Not sure about this, but Google Hangouts uses it
    googCpuOveruseEncodeUsage: true,
    // The CPU % threshold for overuse
    googCpuOveruseThreshold: 70
  }
}

While these have improved it somewhat we're still seeing higher usage than, say, Google Hangouts on equivalent machines (which as far as I understand is using the same underlying browser technologies). Users also complain that performance/quality gets worse as the call goes on (up to 2 hours or so).

I know screen capture and video encoding are expensive, but I'm hoping there's some things I've missed that can improve performance, or ways to debug it. Browser profiling doesn't do too much here because the Javascript isn't actually doing a lot of work, it's the underlying WebRTC and video encoding implementations. If someone could point me in the direction of a Twilio video JS performance guide or something that would be sweet!

[manjeshbhargav]

@gregoryjjb

Sorry for the delayed response. We are working on such a guide and it will be available soon. In the meantime, there are some things you can consider:

1. Are you in a Peer-to-Peer Room? If so, using a Group Room will help since you have to encode and transmit your media only once instead of once per Participant.
2. For screen share, you can capture it at a lower frame rate (15 fps) to save some CPU cycles.
3. I don't know if PCMU will help. Opus should be fine.
4. Maybe 1080p is fine for screen share, but I think you can reduce the capture resolution of the camera to 720p.

One more thing. We are working on a feature called Bandwidth Profiles which will enable you to configure how your downlink bandwidth should be allocated to your subscribed Tracks. I will reach out to you once this is available (probably by end of this week or beginning of early next week). This should also help you with some of the performance issues you are facing.

Thanks,

Manjesh Malavalli
JSDK Team

[gregoryjjb]

Thanks for the tips @manjeshbhargav. We are using group rooms, and most of our calls are 1 on 1 anyway. I'll try switching back to Opus as well. I'll be eagerly awaiting the performance guide and the bandwidth profile feature!

[manjeshbhargav]

@gregoryjjb ,

The Bandwidth Profile API is available with twilio-video.[email protected]. Please let me know if it helps.

Thanks,

Manjesh Malavalli
JSDK Team