Mixedrealitytoolkit-unity: XR Plugin: Camera not being correctly tracked.

[davidkline-ms]

From holodevelopers.slack.com:

Kolodi
Thx. In fact, I have several issues with new XR Plugin. The most important is related to the camera. In editor play model the game view is completely black (while i can clearly see the camera image being rendered correctly when selecting the camera); But the most important is that camera is not being tracked correctly on the device, I can see the Tracked Pose Drive being added during play mode (see image), on device it is tracking somewhat but the camera image is out of focus (very weird effects) and the tracking itself is not precise at all. Se also my XR settings.

https://files.slack.com/files-pri/T1AJLR56G-F01624GB9CM/image.png

https://files.slack.com/files-pri/T1AJLR56G-F0169M132PN/image.png

https://files.slack.com/files-pri/T1AJLR56G-F0163FS26AE/image.png

[kolodi]

After updating everything I could (Unity 2019.4.1f1 LTS, latest XR Plugins) the issue of black preview in play mode in editor is gone. But the tracking and stereo rendering problems are still in place. I have checked all settings I could. Camera and Input profiles are set according to the instruction here. I can see in play mode in unity that there is Tracking Pose Driver added automatically to the camera object. I have tried to add it manually beforehand and played around with different settings. The result is still unusable: stereo rendering is quite off and the tracking is not precise at all. I have also tried to cover the tracking cameras to force loosing tracking, while I can observe some improvements in tracking and rendering, it is still unusable overall.

[Nuno-Monteiro]

Any update on this? I think I'm having the same issue.
It feels the whole experience is generally closer to my face than before (when running on the device).

One thing I noticed and should be quick to sample out is the splash screen of the app itself. It makes me cross my eyes because it is now much closer to me than before.
Other than that, the first screen I show in my app is a custom image I made with four highlighted corners to make sure the user has the device correctly positioned on its head. I can see about 2/3 of this image now ,from the top right corner downwards and to the left.
If I build with spatial mesh visible the further it gets from me the more unstable and unmatching reality it is.
I have the same settings as shown here, reprojection using Depth, and depth at 16bit.

If it's not the same issue please tell me and sorry for hijacking this one I'll create a new one.

Cheers!

[js-dignitas]

I am seeing the stereo issues when running with Holographic Remoting and with the App running directly on the Hololen 2. I've tried

• Windows XR Plugin 2.1.5 and 2.2.0
• XR Plugin Management 3.2.13
• Unity 2019.4.1f1, 2019.4.2.f2
• XR Legact Input Helpers 2.1.4

You can make a simple App with just a cube and no MRTK and you will see this issue.

In this image, the left and right eyes are shown. This is taken directly from the Game preview in Unity using Holographic Remoting. Tracking was off, so the eyes are perfectly level. You can see eyes are shifted vertically

Also interesting, if you look at the orientation of the eyes through the InputDevices, the left and right have pitch. I wonder if this is related...

[amoffatt]

I have the same results as @js-dignitas

[js-dignitas]

@amoffatt Windows XR Plugin v3.3.0 has the fix for Unity 2020.1 and v2.30 for Unity 2019.4.
According to the change log, I imagine "Multiple corrections for hologram stability and convergence" is related.
https://docs.unity3d.com/Packages/[email protected]/changelog/CHANGELOG.html

[keveleigh]

Ah yes, I updated a different issue but didn't post here as well: https://github.com/microsoft/MixedRealityToolkit-Unity/issues/8024#issuecomment-669391603

The x.3 versions of the plugin have a fix that should resolve the offset eye issue on HL2.