Arcore-android-sdk: Feature request : 3D Object Detection

[bilelmnasser]

Hello everybody ,
it will be quiet cool to recognize real objects from a simple 360° scan.
ARkit 2.0 support this :dancer:

@inio @gordonbrander @eitanme @richlowenberg @jsantell

[inio]

Indeed, that's a really neat feature, isn't it?

It's among the many potential additions we're looking at.

[izdi]

Hey @inio
While it's indeed a neat feature I'd like to discuss a little workaround I use.

Using any known object detection neural nets (SSD, YOLO, TF API) we can get in a separate thread objects from a frame, then pass satisfied results, in this case coordinates of a bounding box center, back onto drawing thread and make a hitTest for anchors in that frame region and hopefully attach our 3d object.
While this works OK for some objects on a plain and there are lots of anchors, for objects that lack anchors this won't work.
As I understood correctly at this moment ARCore knows distance to the real world object only via anchor hit calculation.
So I have a question: what could be the workaround if detected object won't have anchors. In my mind it so:

• find the nearest anchor to the region of the real world object (probably if it's standing on a place then it's in the same distance)
• calculate distance this anchor instead
• use calculated distance to create anchor in the region without anchors

Another issue I have with this approach is placing 3d objects without a plane. Andy is inclined because anchor is not on the plane (see the picture):
https://i.stack.imgur.com/JK3Pm.jpg
What is the workaround render Andy by straight angle from the anchor ?

[paragbhagwat3575]

this is a good feature to have and will close the gap with Vuforia 3d object detection

[bryanliujh]

Hopefully arcore supports it, would love to use it.

[tony-topper]

Maybe start with migrating the TensorFlow Object detection example app over to using ARcore. That would be a nice way for people to experiment and give feedback.

[joao-carvalheira]

Any updates on this? Is anyone working on it?

[kinster]

Maybe start with migrating the TensorFlow Object detection example app over to using ARcore. That would be a nice way for people to experiment and give feedback.

Looks like image classification (2D stuff) not Objects

[Lucaffo]

@kinster , @tony-topper might he was referring to this