Emscripten: [WebVR] Use VRDisplay requestAnimationFrame as Emscripten main loop

[Noxalus]

Hello everyone :)

I'm currently trying to use the WebVR API rendering a 3D scene in C++ for a HTML5 application, it works, but the result is a little bit laggy for now.

Comparing to an official WebVR sample, I noticed that the FPS displayed by the Chromium developer console is above 350 FPS when I click on the Enter VR button.

With my app, I'm stuck to 60 FPS, and it seems to be because of Emscripten:

emscripten_set_main_loop(emscriptenMainLoop, 0, 1);

Reading the Emscripten documentation, I saw this:

Setting 0 or a negative value as the fps will instead use the browser’s requestAnimationFrame mechanism to call the main loop function. This is HIGHLY recommended if you are doing rendering, as the browser’s requestAnimationFrame will make sure you render at a proper smooth rate that lines up properly with the browser and monitor.

But the WebVR API provide a specific requestAnimationFrame() function with a higher refresh rate that I should use instead when a HMD is detected.

Do you know if there is a way to say to Emscripten to use another requestAnimationFrame() function during runtime?

Thanks in advance for your help.

[kripken]

I agree, it does look like we should use the new WebVR requestAnimationFrame. I'm not that familiar with src/library_vr.js (the relevant code) or WebVR in general though, ccing some people who might know what the best thing to do is, @cvan @vvuk @line0

[Squareys]

I had a look into this and would propose a emscripten_vr_set_device_main_loop(device, callback) instead of allowing to replace the "classic" emscripten main loop with the VR device main loop.

Mainly because of the following line in the spec of VRDevice.requestAnimationFrame():

the HMD runs asynchronously from other displays and at differing refresh rates

Which should mean that it is possible to render image "A" to the browser window while rendering image "B" to the VRDevice. This can be particularly useful, if "A" is a spectator screen to what is going on in VR.

Additionally, this would keep library_browser untouched and decoupled from library_vr.

@kripken If this sounds sane to you, I will give it a shot and provide a pullrequest. As library_vr is kinda outdated as far as I can tell, how much of its API needs to be preserved?

[Squareys]

I dove into this alot deeper and now have some local changes which add
~c
extern void emscripten_vr_set_display_render_loop(WebVRDeviceId deviceId, em_vr_callback_func callback);
extern void emscripten_vr_set_display_render_loop_arg(WebVRDeviceId deviceId, em_vr_callback_arg_func, void *arg);
extern void emscripten_vr_cancel_display_render_loop(WebVRDeviceId deviceId);
~

and adds some API documentation (which is currently missing entirely).
I started embracing a rename of device to display as navigator.getVRDevices() has been removed in favor of navigator.getVRDisplays(), as VR controllers have been moved to the gamepad API.*

* I would rename this everywhere given that the API does not need to be preserved.

[kripken]

So as mentioned above I don't know much about vr. I suppose the cc'd people are too busy. At least for the API design though hopefully @juj can give feedback.

Overall I'd say if there aren't vr people available to review this, it's still ok to do this if your changes don't impact other things (which sounds like the case).

[juj]

Additionally, this would keep library_browser untouched and decoupled from library_vr.

This is a good idea.

As library_vr is kinda outdated as far as I can tell, how much of its API needs to be preserved?

The library is dated in the era when WebVR API was still being specced and not yet standardized. No need to preserve any bits, since it is not functional with respect to the official spec.

I dove into this alot deeper and now have some local changes which add

There is pressure to do other changes to main loop functions in Emscripten that I have found while creating new multithreaded runtime support, I'll add this requirement to the list and plan these types of changes at one go. However that will probably take more time than your current work, so adding VR-specific render loop functions is ok if we can keep that specific to WebVR libraries.

I started embracing a rename of device to display as navigator.getVRDevices() has been removed in favor of navigator.getVRDisplays(), as VR controllers have been moved to the gamepad API.*

That sounds good. Thanks for working on this!

[promethe42]

@Squareys let me and @Noxalus know when/how we can test a PR and provide feedback. We've an existing Emscripten WebVR ready app.

adding VR-specific render loop functions is ok if we can keep that specific to WebVR libraries.

Cool! Can't wait!

[Squareys]

@promethe42 Thanks for pointing that out, will do. I will likely break the API a bit, since juj seems to be okay with it. I hope that is alright for you, too.

[Squareys]

I opened a pullrequest (#5508), it'd be awesome if you all could have a look at it. I will finish the documentation over the next two days.

[Squareys]

@kripken This one can also be closed :)