Webxr: WebXR API for systems that track users to do XR on a laptop

[IoanaMereuta]

Hi all,
I was wondering if the current WebXR Device API version provides input and output to systems that use eye-tracking to enable users to experience XR without the need for HMD? Basically, our system, simulated reality works as follow:
1- We get eye-tracking data,
2- Use eye-tracking data to calculate projection matrices
3- We render side by side stereo
4- We use a post-processing shader to turn the side by side into the right input for our devices (Laptops, workstations, monitors).
Thanks in advance

[Manishearth]

I believe user agents have the choice to enhance eye pose information with eye tracking info

[cabanier]

Yes, for instance, the Magic Leap system already does this.

[AdaRoseCannon]

Since this is on a laptop, is that going to be an issue? Iirc there was an issue regarding diorama style use cases.

[cabanier]

Since this is on a laptop, is that going to be an issue? Iirc there was an issue regarding diorama style use cases.

I think we need to know a little bit more about the workflow.
Is the user wearing special glasses to capture the eye position and provide stereo?

[IoanaMereuta]

We are using realsense cameras to track the eye position. The experience for the user is glass-free.

[cabanier]

We are using realsense cameras to track the eye position. The experience for the user is glass-free.

How is the side-by-side translated to an image for each eye?

[IoanaMereuta]

On our displays, we have a lenticular overlay that is able to project images separated in space before the display. Furthermore, we have embedded image processing which combines the left and right image and the eye position information to project the correct image through the lenses in each eye.

[cabanier]

Great!
In that case, the WebXR API is designed to handle your case.
We specifically enhanced the projection matrix of the view to account for movement of the eye. This allows you to pass the projection matrices you calculated to the author.

[Manishearth]

Closing: the API already supports this use case, there just are not any user agents that do