Arcore-unity-sdk: ARCore with Cardboard causes objects to not appear to move the proper amount (incorrect FOV)

[JulianCortes]

Anchored objects doesn't have the same precision when built for Cardboard.

[chaosemer]

Could you please provide a video of the issue you are seeing? That'll make it so we can investigate and diagnose.

Thanks!

[JulianCortes]

https://youtu.be/XHEtluTjuzU

Here is a video.
It's been built with Unity 2017.3.0f3 with the Cardboard integrated plugin with ARCore package from here.

The device is a Samsung S8, with ARCore alone it works great, tho.

Thanks.

[bmitchinson]

We're in the middle of a hack-a-thon, hoping to win the AR category, and have the same issue :(

[chaosemer]

Thanks for the video! It looks like Cardboard's lens-driven FOV is conflicting with ARCore's camera-driven FOV setting. This causes objects to move an incorrect amount and appear to not be tracking. This is expected, you'll need to place the ARCore camera image on a plane in 3D to have the rendering appear correct.

I'm updating the title to reflect this. I'm happy to help you out with placing a plane, which is the best you'll be able to do.

[JulianCortes]

@chaosemer So you're saying that i need 2 cameras:

• _ARCore_ rendering on an image texture on a plane.
• _Cardboard_ camera that sees the plane but with a correct FOV.

TIA.

[chaosemer]

You only need the Cardboard camera. You would put the texture for the ARCore Camera image texture on a plane you position in the world.

The size of the plane would be such that the FOV for the image is correct in Cardboard. We don't expose the FOV directly, you'd have to deduce it from the projection matrix returned from Frame.CameraImage.GetCameraProjectionMatrix.

[SFBAWashable]

@chaosemer

Can you please elaborate more on how to add this plane in Unity, and the deduction of the FOV?

Thanks in advance.

[JulianCortes]

Hi,
i botched something working, not fully working since the project i was working got suspended for now.
Anyways:

• Frame.CameraImage.GetCameraProjectionMatrix this function gives you the projection matrix, i feeded it to a camera with Camera.projectionMatrix. The camera i'm using doesn't render anything, it is child of the main camera moved by the Google Cardboard, so remember to set the target display to none, or the camera will be controlled by the VR plugin!!!. I just use it to create the quad from the projection matrix since i'm not that good at math. By giving the camera the projection matrix it will assume the FOV used by ARCore, you will not see the inspector updating, its ok, you can still see the projection fixed in the scene view.

• The only thing is missing is the quad. You can see here how. I used then Camera.ScreenToViewport to get the 4 angles of the screen. If you use the sample on how to create a quad the order would be: bottom left so x = 0, y= 0; then bottom right so x = Screen.width, y = 0; then upper left so x = 0, y = Screen.height; and for last upper right so x = Screen.width, y = Screen.height. The Z depends on how far you want you create the plane, if you want it on the background you can just put the same far plane -1 of the camera.

• The quad is now at the same FOV of the Cardboard camera, it will be smaller, a lot smaller than the screen, i think it's a limitation don't know how to solve that. I then just assigned the same material used by the background renderer of ARCore that can be found on the sample project.

Notes.
The result will be something very similar to this. I still had a couple of issues, but as i said the project got suspended and didn't had the chance to find solutions. The projection matrix given by the function doesn't work correctly if i get it at Start, so for now i hardcoded the values of the Matrix4x4 of the phone, but i think you can update the position of the vertices of the quad every frame. I didn't need to use Camera image texture like suggested above, somehow the shader of ARCore background renderer already updates the texture... didn't went to investigate.

[SFBAWashable]

@JulianCortes Thanks for the detailed reply.

I am a bit confused about the first point. I do not see Target Display option in the camera. This option disappears when I switch build settings to Android platform.

Is there a Cardboard camera object in Unity? I only have camera with ARCoreBackgroundRenderer?

Sorry if my questions are too naive, I am a newbie.

[JulianCortes]

You have to activate the VR part... take look at this, it explains how to implement Google Cardboard on a Unity project.

[jondyne]

@chaosemer, @JulianCortes - trying to make this solution work. Check out https://youtu.be/KFUErLyLrbQ
as you can see I am having tracking issues as the cube seems drift with head rotation. Is this something that is solvable currently or is there an inherit limited somewhere?

[JulianCortes]

@jondyne IIRC even solving the FOV issue, i had the same "issue". The project i was working on was halted and that issue wasn't a gamebreaker.

[smohanty92]

@JulianCortes - thanks for listing your methodology. Would it be possible to list exactly what you did to alleviate the issue? TIA

[jondyne]

We developed a work around in Unity which allows for ARcore experiences to be used in a headset see the implementation here - https://github.com/jondyne/ARcore_mobileVRHeadset/
For a bit of background on the project see - https://medium.com/@jon_54445/augmented-reality-for-mobile-vr-headsets-adc6d5ed11fe

[XaviFields]

@jondyne thanks for that project. It works pretty well.
I noticed it doesn't have ARCoreSessionConfig where I can configure the AugmentedImageDatabase.
Is there another way of setting an image database to work with?

I have an AR project that uses Augmented Images and we want to use VR glasses to be hands-free.

[tomekkie]

@jondyne thanks, works well, any ideas how on how to get this working on GearVR?

[Zumbalamambo]

@JulianCortes Im using Google ARCORE and GVR. I could not be able to find an option with Target Display option. Could you please suggest me how I can enable it?

[JulianCortes]

@Zumbalambo Sorry no, it's been a while since I worked with Unity projects so I'm a bit outdated.

[Zumbalamambo]

@JulianCortes Could you please share with us the code that you have been working on so as to help us understand the underlying method?

[JulianCortes]

@Zumbalamambo I don't have access to that solution anymore.