Mixedrealitytoolkit-unity: Unity UWP Build: Reference Rewriter found some errors

[gilbdev]

Overview

When Building Unity Project for UWP through the Mixed Reality Toolkit>Build Window, the build succeed, but some errors are displayed at the end :

Error: field System.Numerics.Vector3 Windows.Perception.People.HandMeshVertex::Position doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityArticulatedHand::UpdateControllerData(UnityEngine.XR.WSA.Input.InteractionSourceState).
Error: field System.Numerics.Vector3 Windows.Perception.People.HandMeshVertex::Position doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityArticulatedHand::UpdateControllerData(UnityEngine.XR.WSA.Input.InteractionSourceState).
Error: field System.Numerics.Vector3 Windows.Perception.People.HandMeshVertex::Normal doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityArticulatedHand::UpdateControllerData(UnityEngine.XR.WSA.Input.InteractionSourceState).
Error: field System.Numerics.Quaternion Windows.Perception.People.JointPose::Orientation doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityArticulatedHand::UpdateControllerData(UnityEngine.XR.WSA.Input.InteractionSourceState).
Error: field System.Numerics.Vector3 Windows.Perception.People.JointPose::Position doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityArticulatedHand::UpdateControllerData(UnityEngine.XR.WSA.Input.InteractionSourceState).
Error: field System.Numerics.Vector3 Windows.Perception.Spatial.SpatialRay::Origin doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityEyeGazeDataProvider::Update().
Error: field System.Numerics.Vector3 Windows.Perception.Spatial.SpatialRay::Direction doesn't exist in target framework. It is referenced from Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.dll at System.Void Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityEyeGazeDataProvider::Update().

PostProcessWinRT:RunReferenceRewriter() (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWinRT.cs:1062)
PostProcessWinRT:Process() (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/PostProcessWinRT.cs:217)
UnityEditor.BuildPipeline:BuildPlayer(String[], String, BuildTarget, BuildOptions)
Microsoft.MixedReality.Toolkit.Build.Editor.UnityPlayerBuildTools:BuildUnityPlayer(IBuildInfo) (at Assets/MixedRealityToolkit/Utilities/BuildAndDeploy/UnityPlayerBuildTools.cs:115)
Microsoft.MixedReality.Toolkit.Build.Editor.d__2:MoveNext() (at Assets/MixedRealityToolkit/Utilities/BuildAndDeploy/UwpPlayerBuildTools.cs:108)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:Start(<BuildPlayer>d__2&) Microsoft.MixedReality.Toolkit.Build.Editor.UwpPlayerBuildTools:BuildPlayer(UwpBuildInfo, CancellationToken) Microsoft.MixedReality.Toolkit.Build.Editor.<BuildPlayer>d__1:MoveNext() (at Assets/MixedRealityToolkit/Utilities/BuildAndDeploy/UwpPlayerBuildTools.cs:89) System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:Start(d__1&)
Microsoft.MixedReality.Toolkit.Build.Editor.UwpPlayerBuildTools:BuildPlayer(String, Boolean, CancellationToken)
Microsoft.MixedReality.Toolkit.Build.Editor.d__68:MoveNext() (at Assets/MixedRealityToolkit/Utilities/BuildAndDeploy/BuildDeployWindow.cs:1016)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(d__68&)
Microsoft.MixedReality.Toolkit.Build.Editor.BuildDeployWindow:BuildUnityProject()
UnityEditor.EditorApplication:Internal_CallDelayFunctions()

Expected Behavior

No errors

Actual Behavior

Build succeed but erros are displayed

Steps to reproduce

• Create a Project, add MRTK v2RC1, switch to UWP platform, target Hololens x86, backend IL2CPP, Virtual Reality Supported
• Open Mixed Reality Toolkit > Build Window
• click on Build Unity Project

Unity Editor Version

2018.3.11f1

Mixed Reality Toolkit Release Version

v2.0.0 RC1

Visual Studio 2017
Microsoft SDK 10.0.18362

[Peepijeep]

Excactly same problem here.
Even with Scripting Backend set to .NET, I get this error

[andreasherbig]

We can confirm the same issue.
Tried with Unity Editor Version 2018.3.0b2 and 2018.3.12f1 - same result.

The resulting build can be deployed on the Hololens (1), but it seems like there are still issues related to those mentioned errors popping up in the debugger

[davidkline-ms]

Thanks for the report!

[frangagn]

Same problem here! Tried reimporting the MRTK, used completely empty project, still same errors.

[wiwei]

Following up here, we've reached out to Unity folks and filed a Unity bug on this.

https://fogbugz.unity3d.com/default.asp?1146307_k5pfi2qqltmcvqm5

Looks to be some weird interplay between the reference rewriter and not-yet-out-of-insider SDK. The APIs themselves work (i.e. if you get a hololens 2 that supports these APIs they work)

[chrisfromwork]

Assigning to @wiwei for Unity follow ups.

[hanfengsunshine]

https://stackoverflow.com/questions/55687207/how-to-build-and-deploy-a-working-unity-app-with-mrtk-v2-rc-for-hololens-1#

[julenka]

These errors can actually be ignored. Good to leave this issue open for reference until it can be fixed, but we should add notes in getting started docs that these errors can be ignored.

[wiwei]

Clearing the GA blocker tag on this.

We don't have any control over this issue as it's a Unity bug. We can certainly continue to push on Unity to get this fixed, but we cannot make any promises because we have no control over that code.

[wiwei]

@Yoyozilla see last message FYI

[Alexees]

Following up here, we've reached out to Unity folks and filed a Unity bug on this.

https://fogbugz.unity3d.com/default.asp?1146307_k5pfi2qqltmcvqm5

Looks to be some weird interplay between the reference rewriter and not-yet-out-of-insider SDK. The APIs themselves work (i.e. if you get a hololens 2 that supports these APIs they work)

The Unity issue seems to be closed. What was the result of it?

[TautvydasZilys]

It was fixed. The fixes landed to 2019.3.0a6 and 2019.2.0b6 so far, and are on the way to 2019.1 and 2018.4 patches too.

[dfields-msft]

@TautvydasZilys it seems the fix may be incomplete - when evaluated under Mono in the editor, sizeof(System.Numerics.Vector3) returns 16, rather than 12 as expected. This means that using a fixed block to fill a System.Numerics.Vector3[] from native code will (properly) fill in the buffer with a stride of 12 bytes per element, but Mono will index into this memory with a stride of 16 bytes per element. This reproed in 2018.4.6. I suppose we can open a new issue for this, but wanted to comment in the context of this thread for future reference. (PS- This is a general problem, and has nothing to do with MRTK.)

[TautvydasZilys]

@dfields-msft, this issue sounds unrelated. The original issue was merely a flaw in our referenced APIs logic that emitted warnings if it thought that you're calling into something non-existing. What you're describing, however, seems to be a difference in runtime behaviour.

I tried reproducing this using the following code:

using UnityEngine;

public class NumericsTest : MonoBehaviour
{
    unsafe void Start()
    {
        Debug.Log(sizeof(System.Numerics.Vector3));
    }
}

However, it prints 12 in both Unity editor and built UWP player. Do you have any more concrete repro steps (or perhaps a project to share)? It would be great if you could report this issue through the Unity editor - our QA can then dig into it and figure out exactly what makes it happen.

[dfields-msft]

@TautvydasZilys agreed that this seems separate from the original issue, but it may have been masked by that problem. Interesting that you're seeing different results - which version of the editor were you running? I see this in 2018.4.6. I'll also follow up offline and/or through normal QA channels.

[TautvydasZilys]

I was trying a newer Unity version. After talking to our Mono folks, I found this:

https://issuetracker.unity3d.com/issues/system-dot-numerics-vector-sizes-are-all-16-bytes-instead-of-8-12-and-16-bytes

Looks like exactly what you're describing and it explains why I didn't manage to repro. We can backport this to 2018.4. How important/urgent is this?

[dfields-msft]

Aha! Perfect, that's exactly the problem. This is pretty important; when could we expect a backport to 2018.4 to be available, and which versions of 2019 have the fix? (We can totally move this discussion to a more appropriate forum as needed.)

[TautvydasZilys]

You can create thread on Unity forums if you want.

I just cherry-picked the change to 2018.4 branch: https://github.com/Unity-Technologies/mono/commit/33140fdd1d0484ea241552c4d7f0526efc16e84c

Unfortunately, I don't yet know when is the next time Mono changes are going into Unity.

By the way, it will only return 16 inside Unity editor. Built UWP player should work correctly.

[dfields-msft]

Thanks, I posted a reply to the thread in the Unity Issue Tracker link you sent.