Viro: ViroARImageMarker onAnchorUpdated different behavior after update to ViroReact 2.15.0

[natina]

Environment

Please provide the following information about your environment:

1. Development OS: MacOSX
2. Device OS & Version: Android 9
3. Version: ViroReact 2.15.0 and React Native v0.59.3
4. Device(s): Samsung a50

Description

Hi, I'm working on an app that uses Viro React to overlap media icons on recognized image targets. I have a list of targets and each target has a list of icons that have to be shown when the target is on focus on camera. I'm using multiple ViroARImageMarker objects and I'm calling both onAnchorFound and onAnchorUpdated methods. I'm using onAnchorUpdated because onAnchorFound is not called again when I go back to a previous framed target. To understand which is the target current focused on camera, I'm passing a unique target identifier. With Viro React 2.14.0, the onAnchorUpdated method was fired with the correct target ID of the image recognized by the camera at the moment (so it was always the same, and I was calling the render method only when the actual target id passed was different from the old stored in state to update the icons shown on the image). With the new ViroReact version, if I switch from one image to another, the onAnchorUpdated is called repeatedly with the first target id and the second target id (even if the anchored target is the second). So this causes the render called continuously and the app freezes on camera screen. My expectation is that the onAnchorUpdated method takes only the current anchored image like on old version of the library.
It's was expected after the update? Or it's a bug?
Thanks in advance.

[keown]

+1

[nunziofiore]

+1

[achuvm]

Hi @natina,

Thanks for reporting this issue, I'm trying to understand this more clearly. Let me rephrase to see if I understand this correctly, you have multiple ViroARImageMarker components and for each one, you have a listener for the onAnchorFound() and onAnchorUpdated() callbacks. In React-Viro 2.14.0, when a ViroARImageMarker was no longer in the camera view, onAnchorUpdated() was no longer called whereas with React-Viro 2.15.0 onAnchorUpdated() is always called (continuously) even when the image itself is not in the camera view.

If you could maybe provide some example code, that may help me understand better.

Thanks,

[natina]

Hi @achuvm,
thanks for your response. On React-Viro 2.14.0, the method onAnchorUpdated() was called continuously but with the correct target recognized. So it was easy for me to store the last recognized target, compare it with the new recognized target and update the icons that overlap the image. With React-Viro 2.15.0 (I've just updated the library without code changes), the method onAnchorUpdated() is called correctly when I found the first target. After I move the camera and it anchors the second target, onAnchorUpdated() is called from both the first target and the second target continuously. And same for the next targets, it's like the old recognized targets are always on camera (but it's not the case).
Let me post some code. This is inside my render method, where this.targets are the list of images.

<ViroARScene
        ref={(component) => {
          this.sceneRef = component;
        }}
      >
        {_.map(Object.keys(this.targets), (targetKey) => {
          const target = this.targets[targetKey];
          return (
            <ViroARImageMarker
              target={targetKey}
              key={targetKey}
              onAnchorFound={() => this.onAnchorFound(targetKey)}
              onAnchorUpdated={() => this.onAnchorUpdated(targetKey)}
              pauseUpdates={markerFound === targetKey}
            >
              {markerFound === target.name && this.renderArIcons(target)}
            </ViroARImageMarker>
          );
        })}
      </ViroARScene>

This is the onAnchorUpdated method:

onAnchorUpdated(marker) {
    const { markerFound } = this.state;
    if (markerFound !== marker) {
      this.setState({ markerFound: marker });
    }
    clearTimeout(this.timer);
    this.timer = setTimeout(() => {
      this.setState({ markerFound: '', showIcons: false });
    }, 10000);
  }

So basically here I'm just updating the state, so it calls again the render to shown the correct icons on the new target set. With React-Viro 2.14.0, the marker value passed on onAnchorUpdated method, was always the same as the image framed with the camera, even after passing from one target to another. With React-Viro 2.15.0, it's called with the new target found and even with the previously targets found. So this check markerFound !== marker will always be true in this case, and it causes the glitch of the icons, and after some time the app get freeze with a memory error.
Please don't hesitate to ask if you need more explanations.
Thanks,
Natascia

[achuvm]

Hi @natina,

It looks like with the introduction of "TrackingMethod" on the ARAnchors, ARCore now behaves in this way: https://developers.google.com/ar/reference/java/arcore/reference/com/google/ar/core/AugmentedImage.TrackingMethod

Our next update will expose this logic such that when the trackingMethod is tracking then it is "visible" otherwise it should either be notTracking or lastKnownPosition (this property won't be available on iOS)

[natina]

Hi @achuvm,
there is already scheduled a new update? This is a blocker for me, I can't go back to 2.14.0 version because the new version is more accurate showing the icons in the right position on the image target (more like iOS).
Or, do you have a suggestion where I can manually add this new tracking logic?
A sincere thanks, for this amazing library, for the quick answer and you attention for developers issues .

[nunziofiore]

Hi,
exactly the same issue also for me. New version is better for the android experience but there's this trouble of tracking that in previous was ok :|

[fraviloria]

Hi @achuvm @dthian

I have the same issue, there is already scheduled a new update? or please could you help us to figure out it and send us non-official release before the official release.

[achuvm]

Hi all, we're planning on having a release with the update within the next week.

[natina]

Hi @achuvm, thanks for the release. I've updated React-Viro to 2.16.0 version but I'm still experiencing the issue. Maybe I'm doing something wrong, but printing the anchor on onAnchorFound method, always returns this:

{"trackingMethod":"tracking","type":"image","position":[0.16186964511871338,0.21858854591846466,-2.511033535003662],"rotation":[102.86936980653532,-3.467995040458359,-1.6941447538043302],"scale":[1,1,0.9999998807907104],"anchorId":"31289634"}

Can you explain how the trackingMethod property should be set? I tried on ViroARScene but with no luck.
Natascia

[achuvm]

Hi @natina,

When the image marker is no longer in view of the camera then the trackingMethod should be lastKnownPose.

See the ARCore docs here for TrackingMethod: https://developers.google.com/ar/reference/java/arcore/reference/com/google/ar/core/AugmentedImage.TrackingMethod

Thanks,

[ecabrerac]

Hi @achuvm.
is there any way to change the number of times per second the onAnchorUpdate function is executed??

Thanks!

[reinvanimschoot]

@achuvm Sorry to piggy back on this thread but is there any ETA when having trackingMethod available on iOS? At the moment I am not getting the field when calling onAnchorUpdated on my ViroARImageMarker.