Webxr: Add "local" XRFrameOfReference

[Artyom17]

@casseveritt recently brought a good issue up. With all the positional tracking untethered devices, like HoloLens or Lenovo Solo you may need to render UI elements in such way that the user doesn't end up inside of them. Even if you initially render something on the distance from the current user's location, the user still may come "into" the UI element. Of course, you always may say that the experience could be responsible to recalculate the UI elements' positions in the world coordinate system; however, if we rely on each apps' (experience) logic then the behavior will be inconsistent between them.
So the idea is, we don't want apps to have to figure out where the user wants UI-related stuff. We want to have a system query where apps can ask for the current "local origin". And when the user does a long-press of a specific button on a controller to recenter or whatever, what it does is update that local origin.
Since this is a system-level setting, all apps would be working with the same location / orientation. So overlays like UI or panels in the VR space, would draw around where the user has said his / her center is. But that wouldn't change the coordinates that the tracking system sends you by default. So stuff you want to draw around the user, you do by drawing it around the "local" pose. But apps that want to draw stuff in the physical world will need to know about both local space and the room space. By providing a query for local space, we don't require apps to try to come up with their own idea of where the user wants local stuff drawn. And, in particular, we don't have to worry about every app (experience) coming up with different ways to communicate this.

So, the idea is to add 'local' XRFrameOfReference type, where the player's position will be (0,0,0) at the time of pose reset, for example, when user press and hold a specific button on the controller.

Any thoughts on that?

[speigg]

Why not just use the eye-level frame, since the UA should be able to let the user reset that frame?

[casseveritt]

Is the eye level frame always intended to be centered around the user? If "eye-level" is synonymous with the local frame, that could work. (Though the "local" name I think captures what is intended more clearly.)

[speigg]

I agree, I think “local” is a better name for the “eye-level” frame—that’s what we used to call it in our Argon project, anyways. But yes, the “eye-level” frame should generally be “near” the user (and the “stage” frame, if there is one, is positioned right below the “eye-level” frame), while the “head-model” frame follows the user around.

[casseveritt]

Ok, I think the name "local" will be a better match for OpenXR, if that matters.

Stage frames don't have an implicit correspondence to the local frame though. Rather apps have to explicitly ask for their relation (the transform between them). Stage space's origin is always at the center of the stage, with the edges axis aligned.

[speigg]

@casseveritt, you’re right, perhaps only the emulated stage would have an implicit correspondence.

[thetuvix]

The "eye-level"/"local" frame (I like the "local" name better than "eye-level") will be quite useful as the app's base substrate coordinate system for seated experiences, enabling multiple seated pages to all start their experience facing forward at the place the user chooses to sit, rather than baking in any incidental head motion before the session was started. It seems the primary change here from "eye-level" to "local" is whether the UA can choose to set the origin to something other than the session's initial device position.

You specifically mentioned "local" for the scenario of rendering UI, perhaps in non-seated experiences as well. A few questions there:

• If an untethered device wants to display UI near the user, would the app just prefer to have that UI tag along with the user instead? We've generally found HoloLens apps building their user-relative UI with "tag along" behavior to keep the UI controls nearby and in the FOV (example), rather than leaving a sphere of UI controls centered around some canonical place in the room to stand.
• Do you see non-seated apps using "local" as the substrate coordinate system that they'll get their view matrices from each frame? Once the user is standing, a VR app will likely want to render a floor under the user, which is why we generally suggest that both standing-scale and room-scale apps use the "stage" frame of reference (standing-scale apps just don't have bounds and so they tell the user to stand still). Especially in WebXR, where we have stage emulation, would standing-scale apps actually just want some floor-level stage-like coordinate system that is centered near them rather than at the middle of the room?

This tension comes because all other experience scales have their own tailored frame of reference except standing-scale:

| Experience scale | Requirements | WebXR frame of reference |
|----------|----------|----------|
| Orientation-only | Headset orientation (gravity-aligned) | "head-model" |
| Seated-scale | Above, plus headset position relative to zero position | "local" |
| Standing-scale | Above, plus floor | "stage"? (requires user to walk to open area
"local"? (doesn't let app render floor-relative content) |
| Room-scale | Above, plus bounds rectangle for open space | "stage" |
| World-scale | World geometry or free-space anchors to hit-test | "world" (as proposed in anchors/#9) |

Perhaps having a dedicated standing-scale frame of reference (e.g. "local-floor") would let a UA keep standing users where they've decided to operate such experiences, without requiring them walk all the way over to the center of their room's large open "stage" area.

[speigg]

@thetuvix I like your idea, but assuming that both the “local” and “stage” frame are available, isn’t the “local-floor” frame simply derivable from the “local” and “stage” frames? in other words, can’t an app just project the “local” frame onto the horizontal plane described by the “stage” frame in order to find the “local-floor”?

Also, I don’t think “standing-scale” experiences necessarily requires having a defined floor. I think there could be scenarios where it might make sense to have a “local” frame but no “stage” frame (outdoors? When on a slope/stairs?).

[NellWaliczek]

Fixed by #409