Mixedrealitytoolkit-unity: WebGL build

Created on 16 Apr 2020 · 7Comments · Source: microsoft/MixedRealityToolkit-Unity

Hello,

Describe the problem

Many contributors and app testers do not have a mixed reality device, such as a Hololens 2.

Describe the solution you'd like

Would it be possible to allow building in WebGL? The goal is to have the same rendering in the browser as in the Unity editor. A user can then simulate a mixed reality device inside its browser.
I manage to do so by modifying some asmdef files and replacing some "#if UNITY_EDITOR" by "#if UNITY_EDITOR || UNITY_WEBGL". I would like to have this natively integrated into the MRTK.

Additional context

With this feature, you could make scene demonstrations (manipulation handler, solvers, shaders, ...) in the browser without the user having to download Unity, download the MRTK and read the documentation.
For developers like me, we could extend the number of beta users without Hololens 2 required.

Cross Platform Feature Request

Source

Rufus31415

Most helpful comment

@keveleigh With MRTK 2.5, all these problems no longer exist. It is possible to compile most of the example scenes in WebGL. I did it for several scenes here :

https://rufus31415.github.io/webxr/MRTK-HandInteraction/

You could offer WebGL examples directly on your website, right next to the documentation. Everyone could test the new features of MRTK without necessarily installing Unity.

I use this trick a lot so that people who don't have a Hololens can still test features or UX

Rufus31415 on 21 Oct 2020

❤2 🚀1 🎉1

All 7 comments

Definitely an interesting idea! What did you find that you needed to change from #if UNITY_EDITOR to #if UNITY_EDITOR || UNITY_WEBGL?

keveleigh on 16 Apr 2020

I think https://github.com/microsoft/MixedRealityToolkit-Unity/pull/7018 will help with this scenario as well, allowing for keyboard and mouse camera movement in builds outside the editor. Will ship in MRTK 2.4.

keveleigh on 16 Apr 2020

I didn't see #7018. Very interesting. I just tested the development branch and almost nothing is missing for the WebGL compilation to work!

I managed to compile for webgl very easily with the mrtk_development branch. For example, here is the HandInteractionExamples scene :
https://rufus31415.github.io/sandbox/webgl-hololens2-simulation/

To achieve this, here are the changes I have made :

Audio

The following audio files have less than 256 samples, so they are not compatible with WebGL. The error is FSBTool ERROR: Failed encoding audio clip 'E:/MixedRealityToolkit-Unity/Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Slider_Grab.wav' to AAC. Possibly the file is too short. Try to append silence such that the length becomes greater than 256 samples.

Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Slider_Grab.wav
Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Slider_Pass_Notch.wav
Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Slider_Release.wav
Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Tap.wav

AR Foundation package

The namespace UnityEngine.SpatialTracking is unknown in file com.unity.xr.[email protected]\Runtime\AR\ARSessionOrigin.cs
I juste add some #if !UNITY_WEBGL around using and some code and then it was ok.

Weird

I get the following error in my Browser during a call to MixedRealityKeyboardPreview.ApplyShellSolverParameters()
The virtual method SolverHandler.DetachFromCurrentTrackedObject() seems to be unknown
I'll find out what's going on...

Invalid function pointer called with signature 'vii'. Perhaps this is an invalid value (e.g. caused by calling a virtual method on a NULL pointer)? Or calling a function with an incorrect type, which will fail? (it is worth building your source files with -Werror (warnings are errors), as warnings can indicate undefined behavior which can cause this)
An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was:
uncaught exception: abort(158) at jsStackTrace (webgl3.asm.framework.unityweb:694:12)
stackTrace (webgl3.asm.framework.unityweb:708:11)
onAbort@http://10.34.0.23:8080/Build/UnityLoader.js:1039:50
abort (webgl3.asm.framework.unityweb:23116:10)
nullFunc_vii (webgl3.asm.framework.unityweb:16078:7)
b158 (webgl3.asm.code.unityweb:4017516:1)
__ZN18VirtActionInvoker06InvokeEjP12Il2CppObject (webgl3.asm.code.unityweb:3724186:1)
_SolverHandler_RefreshTrackedObject_m3A5EB1B08125E45C04CF98BEE5C9BA2DBE6BD4FA (webgl3.asm.code.unityweb:3737341:1)
_SolverHandler_get_TransformTarget_m3E1932AF88F9C115AC6B681B049866D2CE1F2361 (webgl3.asm.code.unityweb:2893668:1)
_Follow_get_ReferencePosition_m3D3137D6B03AF0E1EDC5D1F138EBE05F7BF80489 (webgl3.asm.code.unityweb:2467189:1)
_Follow_GetReferenceInfo_mEABA58344D33C9760D08C5451A793CE2B058F06B (webgl3.asm.code.unityweb:1229151:1)
_Follow_SolverUpdate_m39B0952412B366B1CD9FE742986198FDFC277F0D (webgl3.asm.code.unityweb:1053440:1)
__ZN18VirtActionInvoker06InvokeEjP12Il2CppObject (webgl3.asm.code.unityweb:3724186:1)
_MixedRealityKeyboardPreview_ApplyShellSolverParameters_mAC71E59553175615C59538E441D4935FE3AA955A (webgl3.asm.code.unityweb:1266045:1)

Mixed Reality Camera Setting Profile

To avoid a warning, the file Assets\MixedRealityToolkit.Providers\WindowsMixedReality\Shared\Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.Shared.asmdef should include plateform WebGL. Related to #7119 and #7669

Rufus31415 on 17 Apr 2020

@Rufus31415 That's awesome! Thanks a ton for that link. Pretty cool to see things running in the browser!

Unfortunate about the ARFoundation package having an issue, but MRTK should work fine in this scenario with that package uninstalled.

Not sure what's going on under the Weird section though! Definitely weird...

@cre8ivepark Do you know if it might be possible to increase the samples in those audio clips to 256 without affecting other platforms?

keveleigh on 18 Apr 2020

@keveleigh With MRTK 2.5, all these problems no longer exist. It is possible to compile most of the example scenes in WebGL. I did it for several scenes here :

https://rufus31415.github.io/webxr/MRTK-HandInteraction/

You could offer WebGL examples directly on your website, right next to the documentation. Everyone could test the new features of MRTK without necessarily installing Unity.

I use this trick a lot so that people who don't have a Hololens can still test features or UX

Rufus31415 on 21 Oct 2020

❤2 🚀1 🎉1

@Rufus31415 Holy COW, this is super super super cool.

I noticed you chose my elastic menu/manipulator demo scene for one of your WebGL examples. It works really well on my end. That is super awesome.

@CDiaz-MS @wiwei @julenka This is really amazing.

Edit: seems like one or two of the elastic examples actually break with this build... Interesting, I wonder if it is a WebGL-compilation-specific issue or if something actually broke in mainline.