Model-viewer: "View in AR" button sometimes only opens fullscreen instead of SceneViewer

[hybridherbst]

Description

The "View in AR" button is correctly shown. However, for some models, pressing it just makes everything glitch around (some onsceen elements appear, some disappear, some just blank out), pressing it again makes the model go fullscreen, but SceneViewer isn't launched.

Live Demo

https://lemon-town.glitch.me/
but also happens with the default astronaut guy sometimes https://model-viewer.glitch.me/
Video of the behaviour:
https://youtu.be/t6uQEcQDnBU

Browser Affected

• [X] Chrome
• [ ] Edge
• [ ] Firefox
• [ ] Helios
• [ ] IE
• [ ] Safari

OS

• [X] Android, latest Chrome
• [ ] iOS
• [ ] Linux
• [ ] MacOS
• [ ] Windows

Versions

happens with both 0.3x and 0.4x
Device is Xiaomi Mi 9, but have also seen on other Android phones

[cdata]

Hey Soraryu, thanks for reporting! I'm tracking another issue that I believe is the same: https://github.com/GoogleWebComponents/model-viewer/issues/692

I'm looking into the issue and will report back here when I have news.

[dayliff-svr]

I am seeing similar behavior on all of my sites that use model-viewer.

In one particularly peculiar case I have a number of buttons that change the source of the model-viewer. I also have a view button that triggers the activateAR() model viewer function. After restarting my device and clearing my cache the SceneViewer will launch consistently for all models if the user interacts with the buttons that change the model source before using the view in AR button. After testing for some time even changing the model viewer source doesn't guarantee the model will launch in the SceneViewer. It should also be noted that after a model is opened in fullscreen mode all subsequent models will also open in fullscreen mode.

[cdata]

@dayliff-svr thanks for confirming the report. We are looking into the issue.

The fundamental problem appears to be that our "fullscreen fallback" that we use when Scene Viewer is not available is conflicting with the launch of Scene Viewer. It leads to a critical failure where Scene Viewer launch is interrupted, and also <model-viewer> fails to go into its fullscreen fallback mode.

I'm working with the Scene Viewer team to diagnose the source of the problem, and will update this issue as we put a fix into place. Thanks!

[hybridherbst]

Note that I was able to get the models that had issues on the Mi 9 to work on a Galaxy S9 and iPad Pro. All 3 are officially supported ARCore devices and there were no system updates on the Mi. Currently installed ARCore version is 1.11 which I verified as working just a couple of days ago with one of the models that is now failing to open at all.

[cdata]

@soraryu Understood. There is more than just the ARCore version to consider, unfortunately.

[mischoko]

I'm having the same problem with both .gltf and .glb objects. The iOS version with .usdz file is working fine tho.

[hybridherbst]

Do you think there might be a connection to Google today renaming ARCore to "Google Services for AR"?
I hope that didn't change the actual package name and accompanying intents... The S9 that worked didn't have that rename update yet while the Mi 9 did as I just found out.

[cdata]

If it were just a matter of changing the intent, we would already have a fix landed. Unfortunately, it's more complicated than that.

There is a related dependency in the Google Search app. A recent update to that app may be the source of the issue.

[cdata]

After further investigation, it appears as though this issue may have been introduced in Chrome 76. If you all can confirm which versions of Chrome you are seeing the issue in, that would be very helpful 🙇 @soraryu @mischoko @dayliff-svr

[mischoko]

@cdata Yup, chrome76 here tested on 3 devices

[dayliff-svr]

@cdata my 2 devices were both using chrome 76.

[hybridherbst]

Yes, can confirm, Chrome 76.0.3809.89

[PawelGIX]

Chrome/76.0.3809.89
Sometimes the AR view starts correctly. But I don't know if there is any rule here.

[cdata]

A quick update for everyone on what I have discovered so far:

• There seems to be a Chrome bug, and I'm preparing a test case and some logs to record it on crbug.com
• Separately, we (specifically @azakus) have come up with a workaround that might restore the previous behavior in m76, even if the Chrome bug isn't fixed. We are testing it now
• If we ship a hot fix for this issue, it will be based on v0.5.0 and be released as v0.5.1, and you can expect it within the next 24 hours

[milesgreen]

Thanks for jumping on this bug so swiftly.

I too noticed the same behaviour. Sounds like you've identified the cause, but I just wanted to add further confirmation:

• I had 3 Android devices all testing fine with the default astronaut example as well as my own project.
• Also multiple iOS devices, all working fine.
• 2 of the Android devices stopped working (both astronaut and my project), in the way described above - model preview loads correctly, tapping AR icon opened fullscreen, but the "View in your space' button did not show and so unable to view in AR).
• 1 Android device still workd.
• iOS devices still worked.
• Upon checking the Chrome version, the 2 devices that stopped working are now running v76.
• The Android device that is still working is still running Chrome v75.

Thanks and good luck.

[cdata]

FYI for those tracking this issue: draft for hotfix is up in #698 . We still need to test iOS to make sure we didn't regress Quick Look.

[cdata]

@milesgreen @soraryu @dayliff-svr @mischoko @PawelGIX @arturomarruffo v0.5.1 has been released and published to npm. You can also get it on unpkg.com.

We only have a handful of devices to test on (and all of my local devices are Pixel phones). If you have time, please upgrade and try it out on your affected devices so that we can make sure that the fix is satisfactory.

Thank you!

[hybridherbst]

What do I have to do to force an update / do I need to wait for some kind of CDN to bubble through? Default astronaut is still broken right now.

Edit: by remixing the default astronaut I saw that it has 0.5.0 hardcoded and thus is still broken. Removing 0.5.0 (and thus always using the latest) makes it work again (I think using the latest should be default for your examples).

I went through all my model-viewer usages and found that (obviously) all that specify a version now are broken, the ones that don't are working. I didn't really pay attention to this in the past but since I usually copy-pasted from the astronaut example all of mine have different hardcoded versions now. Time to update.

[hybridherbst]

Not sure if it's related, but there seems to still be issues with
a) models disappearing after returning from SceneViewer
b) occasional issues with SceneViewer loading with black background instead of white and never being done loading
c) occasional messages from SceneViewer "can't load model, there was an error" on repeated loading of the same model

[cdata]

@soraryu Thanks for checking it out! We generally do update our examples as we release new versions, but in this case I wanted to hear some verification from the folks in this thread in case we needed to explore a second hotfix to address device-specific issues.

As to your other comments:

a) models disappearing after returning from SceneViewer

Yes, this is related to #619 and we will be releasing a fix with v0.6.0 (probably in about a week or two).

b) occasional issues with SceneViewer loading with black background instead of white and never being done loading

This is an interesting issue that I had not heard before. Would you be willing to file an issue with a screenshot so that I can share it with the Scene Viewer team?

c) occasional messages from SceneViewer "can't load model, there was an error" on repeated loading of the same model

This is also new for me. Any test cases you can provide could help the team diagnose. Please file an issue if you come up with one!

[PawelGIX]

Works on my device. I will do more tests tomorrow.

[scanblue-sr]

0.5.1 works on my S8 (tested with Chrome 76 and Samsung Internet 9.4 browser, all updates installed).

[hybridherbst]

0.5.1 works on Xioami Mi 9 and Samsung Galaxy S9+.

[mischoko]

It's working normally again, thanks for the quick fix.

[PawelGIX]

I have this message on one device.
I don't know if this has anything to do with the above, but it previously worked on this AR device.

[cdata]

@PawelGIX looks like Scene Viewer is crashing for some reason. Would you be able to share the device model?

[dayliff-svr]

0.5.1 works on my s8 and s10. I did see soraryu's black screen issue once, but haven't been able to reproduce it. Thanks for the fast fix.

[scanblue-sr]

On my S7 the scene viewer is crashing with:

E/AndroidRuntime(29931): FATAL EXCEPTION: main
E/AndroidRuntime(29931): Process: com.google.ar.core, PID: 29931
E/AndroidRuntime(29931): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.google.ar.core/com.google.ar.core.viewer.IntentForwardActivity}: java.lang.IllegalStateException: Only fullscreen opaque activities can request orientation
E/AndroidRuntime(29931):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2957)
E/AndroidRuntime(29931):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3032)
E/AndroidRuntime(29931):    at android.app.ActivityThread.-wrap11(Unknown Source:0)
E/AndroidRuntime(29931):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1696)
E/AndroidRuntime(29931):    at android.os.Handler.dispatchMessage(Handler.java:105)
E/AndroidRuntime(29931):    at android.os.Looper.loop(Looper.java:164)
E/AndroidRuntime(29931):    at android.app.ActivityThread.main(ActivityThread.java:6944)
E/AndroidRuntime(29931):    at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime(29931):    at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
E/AndroidRuntime(29931):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
E/AndroidRuntime(29931): Caused by: java.lang.IllegalStateException: Only fullscreen opaque activities can request orientation
E/AndroidRuntime(29931):    at android.app.Activity.onCreate(Activity.java:1038)
E/AndroidRuntime(29931):    at aae.onCreate(PG:23)
E/AndroidRuntime(29931):    at fq.onCreate(PG:62)
E/AndroidRuntime(29931):    at oh.onCreate(PG:9)
E/AndroidRuntime(29931):    at com.google.ar.core.viewer.IntentForwardActivity.onCreate(PG:2)
E/AndroidRuntime(29931):    at android.app.Activity.performCreate(Activity.java:7183)
E/AndroidRuntime(29931):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1220)
E/AndroidRuntime(29931):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2910)
E/AndroidRuntime(29931):    ... 9 more

[PawelGIX]

@cdata Samsung Galaxy A5 (2017) SM-A520F

[sulydeni]

I have this message on one device.
I don't know if this has anything to do with the above, but it previously worked on this AR device.

I am using Samsung s7 edge and have the same issue

[sydadder]

We noticed this issue is happening for Android 8 and below. But since there are still a lot of users out in the market with this version would be essential to have this fixed.

[milesgreen]

@milesgreen @soraryu @dayliff-svr @mischoko @PawelGIX @arturomarruffo v0.5.1 has been released and published to npm. You can also get it on unpkg.com.

We only have a handful of devices to test on (and all of my local devices are Pixel phones). If you have time, please upgrade and try it out on your affected devices so that we can make sure that the fix is satisfactory.

Thank you!

Confirming that testing on a Sony XZ3, Model Viewer v0.5.1 has fixed the problem.

I'll update once I can test with more. Thanks.

[vitorbalbio]

I have this message on one device.
I don't know if this has anything to do with the above, but it previously worked on this AR device.

I am using Samsung s7 edge and have the same issue

I have exactly the same using Galaxy S7

[cdata]

@PawelGIX @vitorbalbio @sulydeni thanks for reporting this problem. Since this seems to be very device specific (and also likely a Scene Viewer bug), I'm going to recommend that we follow-up on it in another issue.

For the record, I have filed an internal bug regarding the Scene Viewer crashes reported on various Samsung Galaxy devices. I have also filed https://github.com/GoogleWebComponents/model-viewer/issues/703 to track updates publicly and will update it with additional information as it becomes available.

As for this issue, I'm going to consider the problem resolved for now. Thanks to everyone for your assistance in reporting and testing out the fix, it was super helpful 😊

[AytoMaximo]

Hello!
I faced the same problem and found out a very interesting solution, maybe it can be helpful: android AR works only with source from https link. If you are loading file from http, the fullscreen mode will be forced.

[scanblue-sr]

@AytoMaximo Loading from https is one of the file requirements for models (see: https://developers.google.com/ar/develop/java/scene-viewer )

[yosun]

This issue still seems to occur in Chrome v78 from the latest update on Pixel XL 2

[cdata]

@yosun can you confirm which version of model-viewer you are experiencing this with?

[oninross]

Ditto. Default astronaut showing up as full screen. Checking dev tools i got this:

[cdata]

@oninross @yosun @AytoMaximo and anyone else who comes across this issue: we will always try to help you address your problem, but unless you can provide us with minimal details that help us to reproduce the problem, there is nothing we can do.

Someone please offer the <model-viewer> version and a description of your local usage of the component, and I will be more than happy to look into this issue. Otherwise, I will need to lock the conversation here and ask that a new issue be filed.

[oninross]

@cdata, all good mate. I just ripped it off from the demo site and is now working like a charm :)

[bwilytsch]

Ditto. Default astronaut showing up as full screen. Checking dev tools i got this:

I have the same error whenever I hit the AR button. Validated my models with https://github.khronos.org/glTF-Validator/ and iOS safari works perfectly. I get this error on Android with Chrome 83.0.4099.5.

I am using [email protected] and added all recommended polyfills as well. Every help is highly appreciated!

P.S. Using it within next.js env

[ybinstock]

@cdata

I'm having the same error as seen above.

Here is the demo link - https://ybinstock.github.io/dailySculptureAR/
Here is my code - https://github.com/ybinstock/dailySculptureAR/tree/gh-pages

I'm using https://unpkg.com/@google/model-viewer/dist/model-viewer.js

I'm on Android S7.