Mixedrealitytoolkit-unity: Vision / Tracking - parity with ARFoundation

[xwipeoutx]

Describe the problem

iOS and Android have easy ways to "align the world" - object tracking, face tracking, image markers, QR codes. It's all easy.

I can't do this easily on Microsoft devices like the surface pro or HoloLens, without a 3rd party (eg. Vuforia)

Describe the solution you'd like

I would like device APIs (ideally) or MRTK implementations to do all the things that I can do in my Android and iOS tablet available on Surface Pro-type devices, and rich access to functionality exposed in ARFoundation across these and HoloLenses. Including

• Image Tracking
• Object Tracking
• Face detection / skinning
• AR Support (for devices running windows eg. Surface Pro - esp. 2 in 1)
• QR Code

Ideally these would be accessible via ARFoundation.

Describe alternatives you've considered

I've used Vuforia to cover some of this, but it's clearly dated - compare ARCore to Vuforia for image tracking

Additional context

Mostly, I want to be able to take the code I've already done for ARFoundation, and have it Just Work on 2-in-1 laptops, and the HoloLens

[handyjellyfish]

The 2 in 1 surface devices are potentially great for this sort of work, and it's really unfortunate that MS is lagging in this space that Apple and Google have smothered. Vuforia are the only providers of an AR solution on MS surface devices, and it is lacking. It would be fantastic to see Microsoft work towards feature parity in this area.

Definite +1 request from me.

[davidkline-ms]

Thank you for the feedback @xwipeoutx!

[gilbdev]

definitely one of the must-have features when one's want to do some AR stuff!
+1

[genereddick]

+1

I would add scene / camera scaling to the list

[anders-lundgren]

Great suggestion! I think these are super important features that need to be added ASAP, both to the Surface line of devices and to the HoloLens and/or MRTK. Aside from being extremely useful, it would signal that MS commitment to Mixed Reality is strong and ongoing. From what I gather, it should be fairly straightforward to get a decent implementation in place on the Surface line, since there is Augmented/Mixed Reality support in the 3D Viewer for Windows 10, that would make perfect sense to improve by adding marker support and opening up for third party applications. For the HoloLens, even a port of the NFT feature from the open source toolkit ARToolkit would be helpful to include in MRTK, see https://github.com/qian256/HoloLensARToolKit and https://github.com/artoolkit/ARToolKit5

In the long run, a roadmap that includes full parity compared to competing frameworks, including facial tracking, object recognition, people occlusion, is also something that is needed to keep traction and interest from developers and investors.

[Fangh]

Moreover Unity3D team has written in the last blog post about 2019.3 that ARFoundation is now supported on the HoloLens. So maybe you should contact them and tell them that this is not the case. Because that's giving false hope...

[xwipeoutx]

Hey @davidkline-ms , how this is looking from a planning / prioritisation perspective? Has there been any chatter on the product team for some of these ARFoundation features (image/face/object tracking) to be device-native?