Model-viewer: Ability to place object on vertical plane in AR

[OriginLive]

Description

Hi, i'm trying to use model viewer with ar, however i would like to display pictures, so i would need to match vertical surfaces. Is there a way to achieve that?

so far i've seen this code:

// NOTE: Currently rays will be cast from the middle of the screen.
// Eventually we might use input coordinates for this.
    this.raycaster.setFromCamera({x: 0, y: 0}, this.camera);
    const ray = this.raycaster.ray;
    originArray.set(ray.origin.toArray());
    directionArray.set(ray.direction.toArray());

However i can't find a way to actually switch plane detection.

Browser Affected

All

OS

• [x] Android
• [x] iOS
• [ ] Linux
• [ ] MacOS
• [ ] Windows

Versions

latest

[cdata]

Currently, only horizontal plane placement is supported by both Scene Viewer and Chrome's WebXR-based AR. Sorry!

[p0wl]

Any updates on vertical planes? We would definity switch to model-viewer if it would support vertical planes (need to place an object on a wall).

[cdata]

Hey @p0wl , I just tested out some of the WebXR APIs in Canary (via this demo) and it looks like wall detection is supported. We are currently pushing really hard to clean up our WebXR implementation ahead of the stable release in Chrome 81, but I'm not 100% sure which features will land in stable at that time. If wall detection is in stable, there is a reasonable chance we will support it in our next release.

cc @elalish @bialpio @klausw to comment on the state of things with more authority.

[elalish]

@p0wl @cdata Yes, I don't believe there are any technical blockers there. The main thing is we need to settle on an API to define how the object should be placed on a vertical surface, and when that should be allowed. glTF defines the front of an object as the +Z, so I propose we place the -Z side of the bounding box against the detected wall. Should we enable rotation about the normal? How about the attributes? Maybe webxr-placement-mode, defaulting to 'floor', but you could specify 'wall' or 'floor wall' for both?

[Praddy2009]

Do model viewer supports vertical plane detection now? If not do we have other opensource alternative

[cdata]

@Praddy2009 work is in progress on our side to support it soon. I'm not sure if there are other open source alternatives, but WebXR-based AR has landed in Chrome so hopefully you'll start to see them soon 🙏

[p0wl]

nice! If there is anything we can do to support (like testing, ...) let me know! Thanks for your work!!

[gregbeast]

Hey everyone. Any news about this vertical plane detection feature?
With Arcore we have use : Config.PlaneFindingMode HORIZONTAL_AND_VERTICAL
In model-viewer there will also be this command to enter in the html?
For info with Apple it is possible to enter the plan detection in the .reality file using Reality Composer.

[elalish]

@gregbeast I've been a bit swamped lately, but I would like to get this in. It will require some careful thought around how to make the UX smooth. It would be relatively easy if you can only place on the floor or the walls, but if you can do both at once it will be hard not to make the object jump, since it will be placing different sides of its bounding box against the planes. If you have any thoughts or experiments you've done along these lines, please share them here.

[silvainSayduck]

I'm just jumping in here to point out that this is something that is supported by AR Quick Look since iOS 13 (I think), so maybe that could be a good source for inspiration of the behavior when switching from horizontal to vertical planes.

I think there are indeed a few tricky things, and they were clearly not super smoothed out when first released by Apple (not sure where they're at now, after almost one year):

• contact shadow plane on the floor or wall
• rotation axis is vertical when on the floor, horizontal when on the wall
• when an object has been rotated on a wall (around a horizontal axis), it needs to be reset to flat on the plane when moving it to a horizontal plane (and vice-versa)
• ...

[Mobiliar3D]

Hello,

I'm also looking forward for this feature. Please let us know when there is any news regarding that!

[milesgreen]

+1 for this feature request 👍 This would be massively useful.

As far as implementation goes, here are my thoughts:

• The vast majority of use cases are going to be either/or: either place on floor or place on wall. There's got to be very few cases where the user might need to manipulate between the two.
• I like your suggestion of a web-xr-placement type of attribute for the author to determine where the object should tack to (and, who knows, maybe that opens up scope for future values such as face, hand, body...).
• And if we really want/need to offer the user the ability to stick something on both the wall and the floor, we can provide them with those options in the front end prior to AR: 'View on Floor' / 'View on Wall' buttons that set the aforementioned attribute then trigger AR mode.
• That way, when entering AR mode, a clear prompt can be presented to the user ('Scan Floor' vs 'Scan Wall')
• As neat as it is that iOS AR QuickLook allows user to do either and it 'just works' if you scan the wall, I actually find it a bit clunky if you're trying to position an object on the floor right next to a wall, as it can sometimes jump onto the wall.
• So I say, keep it simple. Have an explicit attribute for which plane. Optimise experience appropriately.
• I wish SceneViewer would also support this, although I know that's not your gig 😉

[arifullahjan]

I was just looking into webxr to allow our users on android place models on wall also. It looks like webxr supports it out of the box and I don't know why this is disabled on purpose here.
https://github.com/google/model-viewer/blob/2775af13256ab2dc77f9bd968c6e326053c2eda9/packages/model-viewer/src/three-components/ARRenderer.ts#L502-L507

I don't know there could be more complexities 👀 to it but I am willing to help if possible.

A good thing is that webxr's behavior is the same as it's on ios quick-look so i think nothing extra then optimizing the UI interaction for vertical placed models.

[elalish]

@arifullahjan PRs most welcome! As I said above, it's only disabled because I haven't had time to make it work properly yet. Making the UX nice always takes more time than it seems like it ought to...

[lampmaker]

Any updates on this topic? Really would like to be able to place objects on the wall..

[elalish]

Thanks for the votes, the priority of this feature is steadily rising. I'm hoping to get this in for v1.4, but it remains to be seen.

[lampmaker]

@elalish thanks for the update! Just to get me calibrated a bit: are we talking days/weeks/months/years? (won't hold it against you :-).

[elalish]

@lampmaker We release roughly on the Chrome schedule (every 6 weeks). But we're very understaffed, so if you want something quickly, please contribute!

[lampmaker]

@elalish I'd love to contribute but it wouldn't make things better and certainly wouldn't speed things up. 😃 By far not enough knowledge here. But if you need some user input or have me test something I'd be happy to help.

[nguyenbs]

How is the progress of this feature?

[Dovatech]

Hello, any good news ? is there any method to place objects on a vertical plane?🙏

[Aubi0ne]

Hello ! Do we have any news or release date for this feature ? I will need it for my current job so if I can help speed up the process in any way please don't hesitate ^^

[elalish]

I am in fact working on this right now, see the draft PR. Still need to fix the shadow. Any feedback you'd like to give is welcome.

[elalish]

Actually, the most useful thing anyone could do would be to donate a model we can use on an example page for this feature. I need a model that's intended to be placed on a wall (has a flat backside) like a wall sconce or something. If you can offer something under a CC-BY or CC0 license, I'd be happy to attribute you on our site. Bonus points if it's pretty.

[lampmaker]

You are welcome to use these files on our site https://kaboomlaser.com/products/test
two orientations for each model

the dog: under CC BY-NC-ND license
https://model3d.shopifycdn.com/models/o/607dc81aa1cd5671/border_coco_2_.glb
https://model3d.shopifycdn.com/models/o/28c58f6138b57a25/border_coco_1_.glb

the tree: C0 license
https://model3d.shopifycdn.com/models/o/3e6915dbe4fa8411/boom_2_.glb
https://model3d.shopifycdn.com/models/o/c3e1d89b43e169fe/boom_3_.glb

[elalish]

@lampmaker Thank you! I'll use the tree as NC is problematic since even though this is open source, it's still a Google project, and Google is certainly commercial. Please let me know if you want something updated in the attribution.

[lampmaker]

No worries,
If you could make sure attribution is to Kaboomlaser or Kaboomlaser.com
that would be great.

Op za 21 nov. 2020 om 02:18 schreef Emmett Lalish <[email protected]

:

@lampmaker https://github.com/lampmaker Thank you! I'll use the tree as
NC is problematic since even though this is open source, it's still a
Google project, and Google is certainly commercial. Please let me know if
you want something updated in the attribution.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/model-viewer/issues/687#issuecomment-731483005,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHTJRJK3CF3OE3FVVRURK3SQ4IPXANCNFSM4IIBJFQA
.

--

Matthijs Keuper*Directeur Invisua Lighting *

mobile: [phone]

https://www.instagram.com/invisua/ Instagram
https://www.instagram.com/invisua/
https://nl.pinterest.com/invisualighting Pinterest
https://nl.pinterest.com/invisualighting
https://www.youtube.com/user/invisua YouTube
https://www.youtube.com/user/invisua
https://www.linkedin.com/company/1205399?trk=prof-exp-company-name
LinkedIn https://www.linkedin.com/in/keuper/
https://www.facebook.com/invisua?fref=ts Facebook
https://www.facebook.com/invisua?fref=ts
https://www.twitter.com/invisua/ Twitter
https://www.twitter.com/invisua/

Subscribe to our newsletter here
http://invisua.us4.list-manage.com/subscribe?u=1a2110a3ed30a51afbc18bf65&id=a902e5cf03

Invisua Lighting BV

Nuenenseweg 167-B23

5667KP Geldrop, The Netherlands
+31 (0)40 - 737 01 90

The information contained in this message may be confidential and legally
protected under applicable law. The message is intended solely for the
addressee(s). If you are not the intended recipient, you are hereby
notified that any use, forwarding, dissemination, or reproduction of this
message is strictly prohibited and may be unlawful. If you are not the
intended recipient, please contact the sender by return e-mail and destroy
all copies of the original message.