Model-viewer: WebXR does not work from inside iFrame

[hybridherbst]

Description

This is a follow-up to an earlier discussion on Spectrum.

When using model-viewer inside an iFrame, WebXR mode just outputs a black screen. There's a brief split-second moment when a camera image is shown on a tiny slice of the screen, but afterwards it just turns black. The request to allow a WebXR session pops up properly if done for the first time.

The iFrame has allow="xr", but that doesn't change the outcome.

Edit: I also found https://github.com/immersive-web/webxr/issues/25#issuecomment-549959055, but allow="xr-spatial-tracking" exhbits exactly the same behaviour.

Live Demo

Open this page in Chrome on Android
(I have removed the repro page for now since the bug has been accepted as bug - I'll happily test again once resolved)
Click the "View in AR" button (the page isn't very responsive, you might have to turn to landscape mode)
Expected: AR model
Actual: black screen

Browser Affected

• [X] Chrome
• [ ] Edge
• [ ] Firefox
• [ ] Helios
• [ ] IE
• [ ] Safari

OS

• [X] Android
• [ ] iOS
• [ ] Linux
• [ ] MacOS
• [ ] Windows

[elalish]

Thanks, I have a repro and some strange console errors; I'll look into it.

[klausw]

I'm still investigating, but it appears that the WebXR DOM Overlay feature doesn't currently work when used within an iframe. I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1101193 to follow up on the Chrome side.

In the meantime, is there a way to disable use of DOM Overlay in model-viewer as a workaround?

[elalish]

@klausw Ugh; I suppose I could try to detect when model-viewer is in an iframe and not enable dom-overlay, but it still means a large part of webXR's advantage is inconsistent. Any idea how long it'll take a Chrome fix to land?

[elalish]

@klausw Since it looks like the real fix will take a while, do you know a good method to enable dom-overlay only if it's not in an iframe?

[klausw]

@klausw Since it looks like the real fix will take a while, do you know a good method to enable dom-overlay only if it's not in an iframe?

The usual idiom seems to be to check if window.parent == window which is false when in an iframe. Careful, accessing window.parent.location in a cross-origin iframe throws a security exception (and crashes Chrome DevTools...).

[ArvindSharda78]

Any updates when this is going to be fixed as the result of AR on screen viewer is different from webXR. Quality of webXR is better than screen-viewer, this bug is stopping me to use it!

[elalish]

@ArvindSharda78 Sorry, this got delayed to Chrome 86, rolling out in about a month.

[milesgreen]

Does WebXR work as expected from within an IFRAME, now that Chrome 86 is out?

[elalish]

@milesgreen Yes, it should. I haven't had a chance to test it extensively myself, so please let us know how it works for you!

[milesgreen]

For me, broadly, yes. It seems to be.
Although I did see some unexpected behaviour when switching out multiple models in WebXR, but I need to do more testing to replicate and isolate (might be something I was doing wrong). Will update if I have anything specific.

[elalish]

Great, I'm closing as fixed in Chrome 86, but feel free to reply if you notice any further problems.

[milesgreen]

I know this is closed already. But just wanted to confirm, in my testing, ModelViewer / WebXR is working great when served from iFrame.
The lingering issue I mentioned above was due to the fact I wasn't listening for beforexrselect event to prevent DOM interactions causing unwanted interaction with WebXR. So it's all good now.

[ArvindSharda78]

Although this has been fixed for chrome, today I tested it and found issue with firefox on iOS (specifically iPad). When you open any iFrame hosted model-viewer in a firefox on iOS and press on AR button it leads USDZ file download. Can you please look in this bug as it leads to user explicitly downloading the model? Is there a way to stop this functionality for Firefox? this is issue is not reproducible on Chrome or Safari on iOS.

[hybridherbst]

@ArvindSharda78 it _might_ be a Firefox issue with USDZ scheme. Does it properly work outside an iFrame there for the same usecase?

[ArvindSharda78]

@hybridherbst , yes it works properly outside iFrame for the same use case.

[hybridherbst]

OK, then I guess back to @elalish who'll probably recommend you to open a bug with Firefox ;)

[elalish]

@ArvindSharda78 This sounds like a version of a bug I thought I'd recently fixed. Firefox doesn't support WebXR or app intents, so the AR button shouldn't be shown in that browser at all. I tried to disable it by sniffing the user agent string, but perhaps I missed a case. Can you paste your user agent string here? And just to be sure, this is using model-viewer v1.3, yes?

[ArvindSharda78]

@elalish , I opened the https://modelviewer.dev/examples/augmentedreality/#ar in iPad. the AR button was not visible on chrome, on Safari and Firefox it was visible
Tested on iOS (iPad)
Chrome - AR button (Not visible)
Safari - AR button (visible) (AR works)
Firefox - AR button (visible) (AR works too, but if it is in iFrame it downloads the USDZ)
the user agent does not give correct results , the code I used : var FxiOS = navigator.userAgent.match("FxiOS");

[elalish]

@ArvindSharda78 Interesting! So Firefox can't launch an app intent on Android, but it can launch Quick Look on iOS. Are you saying the Firefox iOS user agent string is FxiOS? If you can paste the whole user agent string here that would be great.