Viro: Preferred way of resetting arscene

[rafthedeveloper]

Is there a preferred way of resetting the initial camera position of the scene? If I move ten meters to the right and I want that to be my new [0,0,0] for the scene. How should I go about this?

[achuvm]

Hi @rafthedeveloper,

Unfortunately there's no way of doing this on the Viro platform just yet. We'll add this as a feature request on our platform.

However, there might be a workaround, if you use the getCameraOrientationAsync on the Viro(AR)Scene, you can get the current camera orientation and manually offset the position/rotation of your components for now. See the documentation here: https://docs.viromedia.com/docs/viroarscene#methods

A quick question, do you simply want to reset the position or also the position and rotation?

Andy

[dam00n]

In our latest update, ViroReact 2.3 -> Added resetARSession to ViroARSceneNavigator which allows users to reset the AR session (iOS only). See documentation here -> https://docs.viromedia.com/v2.3.0/docs/viroarscenenavigator

Unable to add this for Android yet. Added a feature request to Google's ARCore and will provide Android support once they add functionality required.

[r0bbarnk]

Any updates regarding resetARSession() for android @dam00n ?

[achuvm]

Hi @r0bbarnk,

This hasn't been added by ARCore yet.

Could you describe the situation where you desire this feature? We can see if there's a workaround for that use case.

Thanks,

[jojjesv]

If the tracking state goes from normal to unavailable and back to normal some objects straight out disappear..

[dam00n]

@jojjesv are you seeing this happen frequently. This is expected behavior when tracking gets lost. Or perhaps I am misunderstanding. Can you provide a video or more details?

[r0bbarnk]

@achuvm @dam00n I've made an App that shows a marker( ViroImage ) at the location of a real estate as a billboard/POI. Postition of the marker(ViroImage) is calculated based of the phone's compass, location of phone and location of marker (x: x, y: 0, z: -(z)). Naturally, I want to be able to update the markers when I'm at a new location. I've noticed that if I move to new location and render other markers, they get offset and sometimes are just floating far away.

On iOS I solve this by using resetARSession() before i render new markers. Works perfectly.

Also as @jojjesv mentioned, as soon as the camera have no details (it's too dark or the wall I'm pointing at has just a single color), the markers gets reset to their initial position and stuck there until the camera has enough details. As @jojjesv mentioned sometimes the markers disappear. On iOS they may float a little, but at least they stay kind of where they should if I had enough details initially on render and moved camera to low detailed background. On Android they just snap to default position and freeze as soon as I have a low detailed camera background.

The application I made doesn't care about details in the camera image as I am not using any surface or details in the camera image. The app just cares about what direction I'm pointing at with the camera and the location of phone and markers.

Is there any way to turn off tracking and just use the plain functionality of having ViroImage stuck at a certain direction and position?

[achuvm]

Hi @r0bbarnk,

Can you further describe the case where you have markers that float away? Are you adding new markers and all of them float away? Or only the old ones float away?

The fixed position of objects on the screen is how ARCore deals with their loss of tracking.

Unfortunately we don't have a way to disable tracking but still use the ViroImageMarker as ViroImageMarker requires ARCore.

Thanks,

[r0bbarnk]

@achuvm thank you for responding and helping.

Lets say every marker is a location in the real world (latitude, longitude, altitude). I will use 0 as altitude every time.

Here is a usecase example:
User presses a button to render 5 closest markers given the users location and direction.
Then user moves to a new location 100 m further away and presses the same button, rendering new markers given users new location and direction.
The markers will then be offset by the distance moved while using the app , and the markers will be wrongly displayed 100 m further back.

I am calculating the markers position like you would calculate a point (x,y) in a 2D coordinate system given angle and distance where user is in origo(center). Hence if origo is offset, the calculations doesn't consider this and the points will be wrong.

If I am not mistaken, the ARSceneNavigator keeps track of position in the 3d world. And whilst having the app open and moving the ARSceneNavigator automatically offsets the user's position to maintain the correct position of the markers rendered. then because position can not be reset, newly rendered markers will always be wrong by that offset.

On iOS i use resetARSession(true, true) to eliminate the need of including offsets in my calculations, which i would rather not want to. My app is meant to be used country wide, and keeping track of offsets seems like an unnecessary way of approaching this as it could equally simple as iOS; Just reset the initial position of the navigator.

[achuvm]

Hi @r0bbarnk,

Thanks for that description, we'll consider implementing our own resetARSession for Android in a future release.

However, for your use case, I believe if you simply offset the computed marker positions with the current position of the camera that will have the same effect and you can actually leave all your old markers in their place too. Because we're dealing with simple positions, we can simply add the X and Z values of the camera and your computed positions together w/o needing to store anything (the Camera position already keeps track of how far the user has moved).

See the below code for how to grab and use the current camera orientation/position:

render() {
  return (
    <ViroARScene ref={(scene)=>{this.scene = scene}} >
      ...
    </ViroARScene>
  )
}

calculateMarkerPositions() {
  var markerPosition = ... // calculate using lat/long where the "user" is at (0, 0)
  this.scene.getCameraOrientationAsync().then(
    (orientation)=>{
      var cameraPosition = orientation.position;
      markerPosition = [markerPosition[0] + cameraPosition[0], 0, markerPosition[2] + cameraPosition[2]];
      // markerPosition is now offset by the camera's offset.
    }
  );

}

Let me know if that helps!

Thanks,

[r0bbarnk]

thanks for your response @achuvm . I've tried your solution before and unfortunately the problem persists.
after moving while having the app open and rendering new markers, they will be wrongly placed and appear floating towards the position at which I started the app. Maybe AR is loosing tracking while I'm pointing or something?

I also implemented orientation.rotation in rotationPivot because another problem on android is that markers are off by 30 - 180 degrees. Even if I know that the compass gives me the right value. But this had only small effect to none. In some cases the markers get even more wrongly placed because of this. This worked great on iOS though.

Also, when tracking returns after being unavailable the markers are rotated slightly(30ish degrees) and are floating around. It resets the markers(updates tracking) after about 5 seconds (in most cases) and rotation is back to normal.
Direction(rotationPivot) of markers is never correct though. on iOS they always are.
(I verified the correct compass direction and location before writing this)

Im trying this on a Galaxy S9 (g960f/ds). I will also try it on g935f as I've had lots of other issues with the new S9.

Also, if I have rendered markers and put phone down (naturally this will make tracking unavailable because camera gets dark) and pick it back up and point in other direction than I initially pointed, the markers are offset(or MORE offset) by the exact factor of the rotation I made from putting the phone down to picking it up again. This does not happen on iOS...

I guess this is why I need the resetARSession() for Android also. Tracking is very unstable.

I can provide code if you are interested. BTW I'm not using ViroARImageMarker I'm using ViroImage. Can this be a problem?

I would really like to get this to work because on iOS it works beautifully, always correct and performance is amazing. ARCore seems far behind ARKit to me...

[achuvm]

Hi @r0bbarnk,

We ourselves have observed that drifting does happen on ARCore in suboptimal conditions (ie. low light, feature-less locales). ARCore and ARKit also work best in conditions where objects/features are close (within 5 meters) as opposed to farther away, so that could also affect your app.

The rotation issue you mention is interesting. Are you accounting for the fact that a 0 degree rotation on ARCore is not necessarily due North? The gravityAndHeading value for the worldAlignment prop of ViroARSceneNavigator only works on ARKit (ARCore doesn't support that feature), so you may need to offset your rotations.

From what we see, ARCore/ARKit have issues relocalizing if it has lost tracking and you have moved to a location where it has not been before (there's no way for the systems to "compare" its new location to its old one).

Sure, if you provide code, we can take a look at it and see if there's anything strange going on.

Thanks,

[kevinrodriguez-io]

Why was this closed? No solution was provided.

For now, I'm using

replace(scene: ViroARScene)

For android and

resetARSession(resetTracking: boolean, removeAnchors:boolean)

For iOS

[Arjunmore6673]

replace(scene: ViroARScene)

how to use this can you Please provide solution

[nilde]

this._arScene.replace({ scene: NewScene, passProps: { someProps: this.state.someProps } })

where _arScene is a ref to your ViroARSceneNavigator.
Hope it helps!

[nhatvuminh]

Hi @r0bbarnk,

Thanks for that description, we'll consider implementing our own resetARSession for Android in a future release.

However, for your use case, I believe if you simply offset the computed marker positions with the current position of the camera that will have the same effect and you can actually leave all your old markers in their place too. Because we're dealing with simple positions, we can simply add the X and Z values of the camera and your computed positions together w/o needing to store anything (the Camera position already keeps track of how far the user has moved).

See the below code for how to grab and use the current camera orientation/position:

render() {
  return (
    <ViroARScene ref={(scene)=>{this.scene = scene}} >
      ...
    </ViroARScene>
  )
}

calculateMarkerPositions() {
  var markerPosition = ... // calculate using lat/long where the "user" is at (0, 0)
  this.scene.getCameraOrientationAsync().then(
    (orientation)=>{
      var cameraPosition = orientation.position;
      markerPosition = [markerPosition[0] + cameraPosition[0], 0, markerPosition[2] + cameraPosition[2]];
      // markerPosition is now offset by the camera's offset.
    }
  );

}

Let me know if that helps!

Thanks,

Thanks for help! ^^