Sceneform-android-sdk: SceneView does not send ACTION_POINTER_DOWN

[DrPaulT]

Looks like SceneView is not sending the correct event when a second finger touches the screen. Instead of ACTION_POINTER_DOWN, I am seeing a second ACTION_DOWN motion event.

This is making two-finger gestures difficult to interpret, and I think it is confusing Android's ScaleGestureDetector class. As a workaround, I've set a transparent view on top and am using the events from that to control the scene view.

[dsternfeld7]

So far, I haven't been able to reproduce this issue. I was able to use ScaleGestureDetector using the following code:

    scaleGestureDetector = new ScaleGestureDetector(this, new ScaleGestureDetector.OnScaleGestureListener() {
      @Override
      public boolean onScale(ScaleGestureDetector detector) {
        Log.d("TEST_SCALE", "Scale Span: " + scaleGestureDetector.getCurrentSpan());
        return true;
      }

      @Override
      public boolean onScaleBegin(ScaleGestureDetector detector) {
        return true;
      }

      @Override
      public void onScaleEnd(ScaleGestureDetector detector) {

      }
    });

    arFragment.getArSceneView().getScene().addOnPeekTouchListener(new Scene.OnPeekTouchListener() {
      @Override
      public void onPeekTouch(HitTestResult hitTestResult, MotionEvent motionEvent) {
        scaleGestureDetector.onTouchEvent(motionEvent);
      }
    });

[DrPaulT]

I'm sorry, my report was vague. Thanks for the code using OnPeekTouchListener.

I had actually attached a Scene.OnTouchListener:

sceneView.getScene().setOnTouchListener(this);

Then when this.onSceneTouch was called, there was never any ACTION_POINTER_DOWN event.

[dsternfeld7]

Thanks for the additional info!

I just tested the following code:

    arFragment
        .getArSceneView()
        .getScene()
        .setOnTouchListener(
            new OnTouchListener() {
              @Override
              public boolean onSceneTouch(HitTestResult hitTestResult, MotionEvent motionEvent) {
                if (motionEvent.getActionMasked() != MotionEvent.ACTION_MOVE) {
                  Log.d("TEST", "Motion Event: " + motionEvent.getActionMasked());
                }
                return true;
              }
            });

I was able to see ACTION_POINTER_DOWN events, what android device / android version are you using? Maybe there is an issue specific to some devices here. Also, are you returning true from the touch listener?

[DrPaulT]

I tried your code above (minus the ArFragment) and I get the same results, all fine, thanks. ACTION_POINTER_DOWN events are delivered as expected.

The defect seems to occur when a node containing a renderable is attached to the scene.

You can create a node without a renderable and attach it to the scene, and you will still get APD.

You can create a node and add a renderable but not attach it to the scene, and you will still get APD.

So I'm guessing maybe something changes once there are actually triangles on the screen? The renderable isn't anything special, just an OBJ with a simple diffuse texture.

This defect occurs on both my test devices:

Huawei BTV-W09, Android 7.0, security patch level 1 June 2018
Motorola Moto G6 Plus, Android 8.0.0, security patch level 1 July 2018

Hope this makes it a bit easier to see what is going on. Let me know if you need any more detail.

[dsternfeld7]

Thank you for the additional information, I appreciate it! Unfortunately, I'm still having trouble reproducing it so we must still be doing something differently.

Would you be willing to share your full code and android manifest that reproduces the issue?

Thanks,

Dan

[DrPaulT]

I've prepared a minimal code sample that uses Andy the robot as the model, and the behaviour is still reproducible, but in doing so I have discovered another detail about the defect.

The ACTION_POINTER_DOWN event is delivered as long as the second finger is not over the model. Initially, our model covered most of the screen so all touches would have been within the bounding box of the mesh and APD would not have been sent.

If second-finger touches occur on the mesh, ACTION_POINTER_UP is delivered as expected, but APD is returned as a simple ACTION_DOWN event. This interferes with the gesture detectors.

In my code, I extend Node very simply:

public class AndyNode extends Node {

    public AndyNode(ModelRenderable renderable, Node parent) {
        setRenderable(renderable);
        setParent(parent);
        setLocalPosition(new Vector3(0, -0.1f, 0));
    }
}

My onCreate to set things up is:

protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    sceneView = findViewById(R.id.scene_view);
    sceneView.getScene().setOnTouchListener(this);
    modelNode.setParent(sceneView.getScene());
    Camera camera = sceneView.getScene().getCamera();
    camera.setLocalPosition(new Vector3(0, 0, 0.24f));
    ModelRenderable.builder()
            .setSource(this, Uri.parse("andy.sfb"))
            .build()
            .thenAccept(renderable -> new AndyNode(renderable, modelNode))
            .exceptionally(
                    throwable -> {
                        Log.e(TAG, "Unable to load Renderable.", throwable);
                        return null;
                    });
}

I handle events with:

public boolean onSceneTouch(HitTestResult hitTestResult, MotionEvent motionEvent) {
    if (motionEvent.getActionMasked() != 2) {
        Log.i("TOUCH_EVENT", "Action = " + motionEvent.getActionMasked());
    }
    switch (motionEvent.getActionMasked()) {
        case MotionEvent.ACTION_DOWN:
            break;
        case MotionEvent.ACTION_POINTER_DOWN: // Two-finger gesture.
            Log.i("TOUCH_EVENT", "Action pointer down");
            break;
        case MotionEvent.ACTION_MOVE:
            break;
        case MotionEvent.ACTION_POINTER_UP: // Two-finger gesture may end.
            Log.i("TOUCH_EVENT", "Action pointer up");
            break;
        case MotionEvent.ACTION_UP:
            break;
    }
    return true;
}

Let me know if you need an archive of the project, happy to send it over.

[dsternfeld7]

Thank you for all of the additional info! I'm able to reproduce the issue now, it is definitely a bug.

[dsternfeld7]

This is fixed in an upcoming release.

[claywilkinson]

Fixed in 1.6