Viro: ViroImage onClick is not working correctly since updating iOS version to iOS 13

[yangdeloitte]

• [x] Review the documentation: https://docs.viromedia.com/
• [x] Search for existing issues: https://github.com/viromedia/viro
• [x] Use the latest ViroReact release: https://docs.viromedia.com/docs/releases

Environment

Please provide the following information about your environment:

1. Development OS: Mac
2. Device OS & Version: iOS version 13.2.3
3. Version: ViroReact version 2.14 and React Native version 59.9
4. Device(s): iPhone XR

Description

I am displaying a ViroImage when an image is detected. When clicking on the image, it should open a URL on Safari. This worked fine before updating to iOS 13. However, now onClick does not seem to work. It will trigger occasionally after clicking repeatedly. Running the exact same code on Android (Galaxy S10) device and the onClick works fine.

[hokaiyipFS]

I tried both ViroReact version 2.14.0 and ViroReact version 2.17.0.
onClick event on viro-image or viro-node trigger occasionally after clicking.
any clues?

[yangdeloitte]

update: I found out that it actually triggers the onClick when you do a swipe gesture instead of clicking...which is really strange.

[hokaiyipFS]

on true.. hm... it is weird. if I swipe up, it works fine, but it is not a onClick!

[jacklj]

I've had a look through some of the native code but can't spot any obvious places where the onClick and onSwipe events are being confused. At the moment, on the latest iOS it's impossible to click on any AR objects, so ViroReact is fairly unusable... any updates from the maintainers? Thanks

[MisterPhoton]

We are experiencing the same issue - Android is working as intended, but iOS is not detecting presses/clicks as expected. We've found that a swipe/rub/hard press seems to work most of the time, but not all of the time.

[tomascg2309]

Hey! we are experiencing exactly the same and I have 2 things to add:

1) When running directly the AR scene (I mean, without using a homepage or login - whatever) this worked for me:

Original code:

<Viro3DObject
  source={models.button2}
  resources={[materials.button2]}
  type="OBJ"
  position={[-0.01, 0.20, -0.17]}
  onClick={this._submitOption}
  animation={{
    name: this.state.centerButtonAnimationName,
    run: this.state.animateCenterButton,
    onFinish: this._onCenterButtonFinish
  }}
  onLoadEnd={this._onFinishLoading.bind(this, "button")}
/>

New code:

<ViroNode
  position={[-0.01, 0.20, -0.17]}
  animation={{
    name: this.state.centerButtonAnimationName,
    run: this.state.animateCenterButton,
    onFinish: this._onCenterButtonFinish
  }}
>
  <Viro3DObject
    source={models.button2}
    resources={[materials.button2]}
    type="OBJ"
    onClick={this._submitOption}
    onLoadEnd={this._onFinishLoading.bind(this, "button")}
  />
</ViroNode>

The intention of this change is to delegate the animation action to the wrapper (_ViroNode_) and just use the _Viro3DObject_ (or _ViroImage_) to listen the _onClick_ event. I decided to do this since I was aware that it is not possible to use a _ViroQuad_ as a textured object and a shadow receiver at the same time - I had to use 2 different components in the same position.

2) When using a previous component like a home page, everything goes crazy :cry: (all of you know the story: click doesn't work, swipe does) . So exploring a little bit, I added a click listener to the full _ViroARScene_ component and catching the event returned [ ] :thinking: no hints ... playing a little bit more, I could discover if you place the object near the top left corner of the screen and click, IT WORKS... weird, right? Maybe it is an issue related with the "touchable" area of the scene or its positioning when mounting after another component (?) I'm not sure, but just wanted to share this specific discovery in case someone could figure out was is going on while we keep doing our best.

Cheers!
Tomás.

[hokaiyipFS]

@tomascg2309
you are right on about

place the object near the top left corner of the screen and click

very weird indeed.

[tomascg2309]

Good news!
I figured out how to solve it:

1) Make next changes into the AR Component (I'm using a 3DObject, it could be any other). Objective: delegate animations to _ViroNode_ and use objects just to listen events. It will improve your components usage.

Original code:

<Viro3DObject
  source={models.button2}
  resources={[materials.button2]}
  type="OBJ"
  position={[-0.01, 0.20, -0.17]}
  onClick={this._submitOption}
  animation={{
    name: this.state.centerButtonAnimationName,
    run: this.state.animateCenterButton,
    onFinish: this._onCenterButtonFinish
  }}
  onLoadEnd={this._onFinishLoading.bind(this, "button")}
/>

New code:

<ViroNode
  position={[-0.01, 0.20, -0.17]}
  animation={{
    name: this.state.centerButtonAnimationName,
    run: this.state.animateCenterButton,
    onFinish: this._onCenterButtonFinish
  }}
>
  <Viro3DObject
    source={models.button2}
    resources={[materials.button2]}
    type="OBJ"
    onClick={this._submitOption}
    onLoadEnd={this._onFinishLoading.bind(this, "button")}
  />
</ViroNode>

2) Wrap the _ARSceneNavigator_ into a full-screen transparent react-native <View/> to capture correctly the touch event. I did this:

<View style={{ position: "absolute", left: 0, right: 0, top: 0, bottom: 0, width: "100%", height:"100%" }}>
  <ViroARSceneNavigator
    {...this.state.sharedProps}
    initialScene={{ scene: InitialARScene }}
    viroAppProps={{
      ... // Any props to pass to the ARScene
    }}
  />
</View>

Now, my application works amazingly smooth and as supposed to.
Let me know if it works for you!
Good luck y'all.

[hokaiyipFS]

Hi @tomascg2309 ,

I have changed my ARSceneNavigator like you suggested, and also my scene is now very simple:

```
ref={(scene) => {
this.scene = scene
}}

  >
    <ViroBox position={[0, 0, 0]}
             scale={[0.1, 0.1, 0.1]}

             onClick={() => {
               Alert.alert("2")
             }}/>


  </ViroARScene>

```

It is still not working as intended :(

[hokaiyipFS]

 <ViroARScene onTrackingUpdated={this._onInitialized}
                   ref={(scene) => {
                     this.scene = scene
                   }}

      >
        <ViroAmbientLight color="#FFFFFF"/>

        <ViroNode
          position={[0, 0, 0]} >


          <ViroBox
                scale={[0.1, 0.1, 0.1]}
                   onClick={() => {
                     Alert.alert("2")
                   }}/>
        </ViroNode>

      </ViroARScene>

this piece produces the same behavior, I need to put the box to top left to enable onclick

[tomascg2309]

@hokaiyipFS I'm not using any ref at all, maybe it might be messing things up :thinking: (I've set a new project to test my last solution and it is still working)
If it doesn't work, could you show me how your App.js render method looks?

[hokaiyipFS]

@tomascg2309
oh.. I am using ViroReact version 2.17. I have compared the 2 versions, it doesnt seems to have impacted on the way AR object interaction. let me down grade to see as well.

[tomascg2309]

hey! I didn't mean the ViroReact version lol, my idea was using:

 <ViroARScene onTrackingUpdated={this._onInitialized}>

instead of:

<ViroARScene onTrackingUpdated={this._onInitialized}
                   ref={(scene) => {
                     this.scene = scene
                   }}

      >

Extra: I'm using Viro 2.17.0 and RN 0.69.9 (remember the compatibility for each version of Viro)

[hokaiyipFS]

@tomascg2309
I am using exactly your version. without ref, still no luck :(

[jacklj]

@tomascg2309 that fix works for me, nice one, thanks!

Only issue is I'm animating the material of my <Viro3DObject>, and if I move the animations up to the <ViroNode> parent, I can't animate the material any more :(

[tomascg2309]

@hokaiyipFS do you have a public repo so I can check (and try) your code?

[tomascg2309]

@jacklj I know, working with material animations won't work, but if invoking animations from your object works fine (and the click thing still goes well), go for it 😄 fun fact: when I was experimenting with material animations, my android testbed broke 😞

[hokaiyipFS]

@tomascg2309
I think I have resolved it using your way! cheers!

[Hless]

@tomascg2309
I'm not quite sure what the purpose is of wrapping an Viro3DObject into a ViroNode.
Nevertheless, it appears that just wrapping the _ARSceneNavigator_ in a full screen absolute positioned view fixes the issues for us. Cheers!

[quangduz]

"react-viro": "2.17.0",
"react-native": "0.59.9",

onSwipe don't work (click event ok) on any API (ViroNode, ViroImage, ViroBox, Viro3DObject ...), if i swipe, it will fire a click event.
Tried with stand alone or wrap it in ViroNode ....

Sorry for my bad english !

[teabow]

Hi,

I'm facing this strange behavior in my app from time to time (no clicks triggered, swipe ok, left corner trick), it seems pretty random as it's always the same navigation process (i'm using a stack navigator with react-navigation).

I'm able to reproduce this problem for sure with the example below (using react-navigation again) :

@tomascg2309 would you have any clue ?

export const ARTestScreen = () => {
    const [sceneReady, setSceneReady] = useState(false);

    useFocusEffect(
        useCallback(() => {
            const task = InteractionManager.runAfterInteractions();
            task.then(() => setSceneReady(true));
            return () => task.cancel();
        }, [])
    );

    return (
        <View style={styles.fullScreen}>
            {sceneReady && (
                <ViroARSceneNavigator
                    initialScene={{
                        scene: TestScene
                    }}
                />
            )}
        </View>
    );
};

export const TestScene = () => {
    return (
        <ViroARScene>
            <ViroText text="Hello" textAlign="center" textAlignVertical="center" />
            <ViroSphere position={[0, 0, 0]} scale={[0.1, 0.1, 0.1]} onClick={() => Alert.alert('Sphere 1')} />
            <ViroSphere position={[1, 1, 1]} scale={[0.2, 0.2, 0.2]} onClick={() => Alert.alert('Sphere 2')} />
            <ViroSphere position={[2, 2, 2]} scale={[0.3, 0.3, 0.3]} onClick={() => Alert.alert('Sphere 3')} />
        </ViroARScene>
    );
};

const styles = StyleSheet.create({
    fullScreen: {
        position: 'absolute',
        left: 0,
        right: 0,
        top: 0,
        bottom: 0,
        width: '100%',
        height: '100%'
    }
});

[OmkoBass]

Have the same problem, drag event works, but pinch and click don't. Click only works if i swipe the object.

ARSceneNavigator

return <React.Fragment>
        <ViroARSceneNavigator
            initialScene={{scene: ARScene}}
        />

        <View>
            <ObjectList />
            <Button
                title={"Back"}
                onPress={() => history.goBack()}
            />
        </View>
    </React.Fragment>

AR Scene

<ViroARScene onTrackingUpdated={handleTrackingUpdate}>
        {
            loading !== 3
                ?
                <ViroText
                    text={"Initializing AR..."}
                    position={[0, 0, -2]}
                />
                :
                <>
                    <ViroAmbientLight color="#FFFFFF" />
                    <Viro3DObject
                        type="OBJ"
                        source={require("../Assets/Heart.obj")}
                        resources={[require("../Assets/Heart.mtl")]}
                        position={position}
                        scale={scale}
                        onDrag={handleDrag}
                        onPinch={handlePinch}
                    />
                </>
        }
    </ViroARScene>

[OmkoBass]

SOLVED

Here's how i fixed it.

1. Wrap everything inside a React.Fragment instead of View
2. Create a View and add this style to it " style={{ position: "absolute", left: 0, right: 0, top: 0, bottom: 0, width: "100%", height:"100%" }} "
3. Put the ViroARSceneNavigator inside it
4. Then if you have anything else with it create another View and place the other things inside it

<React.Fragment>
        <View style={{ position: "absolute", left: 0, right: 0, top: 0, bottom: 0, width: "100%", height:"100%" }}>
            <ViroARSceneNavigator
                initialScene={{scene: ARScene}}
            />

            <View>
                <ObjectList />
                <Button
                    title={"Back"}
                    onPress={() => history.goBack()}
                />
            </View>
     </View>
</React.Fragment>

Hope this helps!

[developBeetSoftD1]

By editing the viro core code, iss is fixed
this my git soure https://github.com/developBeetSoftD1/react-viro.
Hope to be of help to someone here.

[MartinMasek]

@developBeetSoftD1
Do you think you could create a PR with the fix to the new ViroCommunity? This repo is not maintained anymore

[calebdre]

I was having this problem too - turns out that conditionally rendering ViroARSceneNavigator messes up the click events for whatever reason.

Don't do:

<View
    style={{
        ...StyleSheet.absoluteFill,
        width: "100%",
        height: "100%"
    }}
>
    {condition && (
        <ViroARSceneNavigator
            style={{ flex: 1, height: '100%', width: '100%' }}
            initialScene={{
                scene: Scene
            }}
        />
     )}

     {/* <otherviews> */}
</View>

Do:

<View
    style={{
        ...StyleSheet.absoluteFill,
        width: "100%",
        height: "100%"
    }}
>
    <ViroARSceneNavigator
        style={{ flex: 1, height: '100%', width: '100%' }}
        initialScene={{
            scene: Scene
        }}
    />

    {/* other views */}
    <View
        pointerEvents={'none'}
        style={{
            top: 30,
            left: 30
         }}
        >
              {/*  unclickable content on top of scene  */}
       </View>
</View>