Arfoundation-samples: Black screen when pausing the AR Session

[mehdikhadir]

Hello,

I'm currently trying out the Simple AR sample scene and have noticed that the background camera image turns to black when hitting the "pause" button. Is there a way of actually keeping the last video frame from the camera when pausing the AR session?

Thanks in advance for your help!

Mehdi

[mdurand42]

We have reproduced this on ARCore using the 4.0 packages.'

Can you provide some more information about what packages you are using and whether you are seeing this on iOS as well?

[mehdikhadir]

Hello,

So I'm basically also using the package 4.0. I'm running that sample on a Huawei P20 running EMUI 10.0.0.
I'm mostly interested in Android for now, but I will try with iOS later today when I have some time and post the results here.

Thanks for your help!

[mehdikhadir]

I have tried it on iOS now (same AR Foundation version 4.0) and it's working as expected there, as in the last camera image is displayed when I hit Pause in the Simple AR sample. So it really seems to be an ARCore issue only.

[mdurand42]

Thanks for the additional testing. We have spoken with the ARCore team and they are doing a bit of investigation on their end. We believe this behavior is a byproduct of the new ARCore APIs we began using for more efficient background rendering.

[mehdikhadir]

@mdurand42 Thanks for having spoken with the ARCore team about it, do you know if there is any news regarding the topic?
Thanks for your help

[mdurand42]

We just chatted with Google this morning and we are interested to know your use case for having the background image while the session is paused.

[mehdikhadir]

We basically have an application where the user can freeze the screen so that he can put the device on a table and still be able to see like a "screenshot" of the AR experience he was having. The advantage of this is to keep having the virtual content animated and in place on top of the frozen real background. This way, the user doesn't have to keep holding the device and can work on something else with his hands while still being able to have a look at the AR screenshot.

[tdmowrer]

This was basically always undefined behavior in ARCore, and it has changed with a recent update. However, I think we should be able to preserve the previous behavior in ARFoundation and treat this as a bug.

[tdmowrer]

So this turned out to be a lot less trivial than I was hoping. ARCore now blanks the external OES texture that use to render the background, so preserving the original behavior seamlessly would mean creating a copy of the texture and then using a completely different shader while the session is paused. That's a pretty big refactor, and much more than a "bug fix".

However, if you want to accomplish the use case you described, you can probably do it yourself by saving off the last rendered frame and then drawing that yourself. Something similar was discussed in a recent issue that may be helpful.

[mehdikhadir]

Hi @tdmowrer, thanks for your help!
I see, in this case showing the last frame ourselves would make more sense in that case then.

[TarasViter]

Hi guys!
Have kind of similar issue.
In our case, black screen appears (not every time, but frequently) after you used AR, pause application, by switching to another app or just collapse the app (keep it running in background), and after you open the app again, there will be no camera feed in AR but black screen.
There is only one warning in logcat, but it appears even if "blackscreen" is not appearing:
An input device ARCore with the TrackedDevice characteristic was registered but the ARPoseDriver is already consuming data from ARCore.
We are using 2020.1.0.b4 Unity version
AR core XR plugin, AR foundation, AR subsystem - all are 3.1.0 preview 7.
Hope this information will be usefull.

[tdmowrer]

@TarasViter This sounds like a different issue. You are also using beta & preview versions of 2020.1 & ARFoundation 3.1 but non-beta, non-preview versions of those are available. Please upgrade to a stable release and, if the issue still persists, create a separate issue.

[tdmowrer]

Marking as "wontfix" as this is a change in behavior in newer versions of ARCore.