Sceneform-android-sdk: TransformationSystem when using Sceneform without ARCore

[sandeepbangalore]

TransformationSystem exist for ArFragment which helps move renderables in the scene. But, now that SceneView can be used to create a 3D environment without camera and ARCore session, are there any functionalities similar to TransformationSystem that can help move the renderables around with ease?

[pilgr]

I have the similar issue. I've been trying to setup TransformationSystem in such way

        TransformationSystem ts = new TransformationSystem(getResources().getDisplayMetrics(), new FootprintSelectionVisualizer());

        transNode = new TransformableNode(ts);
        transNode.setParent(sceneView.getScene());
        transNode.select();

        andyNode = new Node();
        andyNode.setRenderable(andyRenderable);
        andyNode.setLocalPosition(new Vector3(0, -0.1f, 0f));
        andyNode.setParent(transNode);

        sceneView.getScene().addOnPeekTouchListener(new Scene.OnPeekTouchListener() {
            @Override
            public void onPeekTouch(HitTestResult hitTestResult, MotionEvent motionEvent) {
                ts.onTouch(hitTestResult, motionEvent);
            }
        });

but when I trying to move the model, I get exception

java.lang.ClassCastException: com.google.ar.sceneform.SceneView cannot be cast to com.google.ar.sceneform.ArSceneView
        at com.google.ar.sceneform.ux.TranslationController.onContinueTransformation(TranslationController.java:122)
        at com.google.ar.sceneform.ux.TranslationController.onContinueTransformation(TranslationController.java:44)
        at com.google.ar.sceneform.ux.BaseTransformationController.onUpdated(BaseTransformationController.java:111)
        at com.google.ar.sceneform.ux.BaseGesture.dispatchUpdateEvent(BaseGesture.java:130)
        at com.google.ar.sceneform.ux.BaseGesture.onTouch(BaseGesture.java:93)
        at com.google.ar.sceneform.ux.BaseGestureRecognizer.onTouch(BaseGestureRecognizer.java:67)
        at com.google.ar.sceneform.ux.TransformationSystem.onTouch(TransformationSystem.java:173)
        at com.google.ar.sceneform.samples.hellosceneform.HelloSceneformActivity$3.onPeekTouch(HelloSceneformActivity.java:193)
        at com.google.ar.sceneform.TouchEventSystem.onTouchEvent(Unknown Source:42)
        at com.google.ar.sceneform.Scene.onTouchEvent(Unknown Source:11)
        at com.google.ar.sceneform.SceneView.onTouchEvent(Unknown Source:9)
        at android.view.View.dispatchTouchEvent(View.java:12513)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3030)
        at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2719)
        at com.android.internal.policy.DecorView.superDispatchTouchEvent(DecorView.java:440)
        at com.android.internal.policy.PhoneWindow.superDispatchTouchEvent(PhoneWindow.java:1830)
        at android.app.Activity.dispatchTouchEvent(Activity.java:3400)
        at android.support.v7.view.WindowCallbackWrapper.dispatchTouchEvent(WindowCallbackWrapper.java:68)
        at com.android.internal.policy.DecorView.dispatchTouchEvent(DecorView.java:398)
        at android.view.View.dispatchPointerEvent(View.java:12752)
        at android.view.ViewRootImpl$ViewPostImeInputStage.processPointerEvent(ViewRootImpl.java:5107)
        at android.view.ViewRootImpl$ViewPostImeInputStage.onProcess(ViewRootImpl.java:4910)
        at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:4427)
        at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:4480)
        at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:4446)
        at android.view.ViewRootImpl$AsyncInputStage.forward(ViewRootImpl.java:4586)
        at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:4454)
        at android.view.ViewRootImpl$AsyncInputStage.apply(ViewRootImpl.java:4643)
        at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:4427)
        at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:4480)
        at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:4446)
        at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:4454)
        at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:4427)
        at android.view.ViewRootImpl.deliverInputEvent(ViewRootImpl.java:7093)
        at android.view.ViewRootImpl.doProcessInputEvents(ViewRootImpl.java:7062)
        at android.view.ViewRootImpl.enqueueInputEvent(ViewRootImpl.java:7023)
        at android.view.ViewRootImpl$WindowInputEventReceiver.onInputEvent(ViewRootImpl.java:7196)
        at android.view.InputEventReceiver.dispatchInputEvent(InputEventReceiver.java:186)
09-26 01:00:28.514 28382-28382/com.google.ar.sceneform.samples.hellosceneform E/AndroidRuntime:     at android.view.InputEventReceiver.nativeConsumeBatchedInputEvents(Native Method)
        at android.view.InputEventReceiver.consumeBatchedInputEvents(InputEventReceiver.java:177)
        at android.view.ViewRootImpl.doConsumeBatchedInput(ViewRootImpl.java:7167)
        at android.view.ViewRootImpl$ConsumeBatchedInputRunnable.run(ViewRootImpl.java:7219)
        at android.view.Choreographer$CallbackRecord.run(Choreographer.java:949)
        at android.view.Choreographer.doCallbacks(Choreographer.java:761)
        at android.view.Choreographer.doFrame(Choreographer.java:690)
        at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:935)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

obviously, changing SceneView to ArSceneView doesn't solve the issue, since ArSceneView requires an ArFragment.

[abramyan]

We've published the source for the ux package where TransformationSystem lives. Hopefully you can customize it for your needs.

Just an FYI, ArSceneView does not require an ArFragment. You can use it on its own but must supply an ARCore Session.

[pilgr]

Thanks, I'll try it out!

[Meenners]

@pilgr have you been able to move forward with this? I'm trying to accomplish the same goal and i cant seem to get things rolling even with the ux package available. So, if you have, any help would be amazing :) Thanks!

[pilgr]

@Meenners not at the moment, been focused on other stuff. But here https://github.com/google-ar/sceneform-android-sdk/issues/336 Romain specified the minimum system requirements to use SceneView. My issue apparently happened because of running an app on not supported ABI. But I have to double check it later.

[dsternfeld7]

The TransformationSystem UX controls are specifically for building AR experiences, as a result they interact directly with ArCore's API, particularly for planes.

You may be able to modify/extend the transformation system to work outside of Ar. I believe the only transformation controller that uses ArCore directly is the TranslationController, so if you remove the TranslationController or write your own that isn't based on moving along Ar Planes then you may be able to build something similar for non-ar experiences.

[Meenners]

Actually looking at the solarSystem example i began thinking about it a little differently.
Getting the MotionEvent from the scene/node itself was good enough for me and do my transformations from this point on...

//{scene/node}
scene.setOnTouchListener((HitTestResult hitTestResult, MotionEvent event) -> { 
//listen and do stuffs to the models through the event 
});

Thanks for the help!

[sarahthe]

I am using the SceneView (without AR) to render a 3D model. As I have an AR view, I tried to use the Transformation System from the AR fragment I have to create a Transformable Node. However, this does not work, I can't transform the model as I can in the AR view. Any ideas? Or do I have to create my own gesture listener instead of reusing a transformation system I have?

sceneView = v.findViewById(R.id.scene_view);
TransformationSystem ts;
ts = MainActivity.getArFragment().getTransformationSystem();
sceneView.setBackgroundColor(getResources().getColor(R.color.colorPrimaryDark));

// create renderable...

TransformableNode node = new TransformableNode(ts);
node.setRenderable(renderable);
node.getScaleController().setMinScale(0.1f);
node.getScaleController().setMaxScale(1.0f);
node.setLocalScale(exhibit.getModelScale());
node.setLocalPosition(exhibit.getModelPosition());
node.setParent(sceneView.getScene());

node.getRotationController().setEnabled(true);
node.getScaleController().setEnabled(true);

sceneView.getScene().addChild(node);
node.select();

[civilordergone]

Any updates?

[civilordergone]

Got it working for the short term. Will make a longer term fix but @pilgr .. I noticed the gesture-listener, was sometimes being added in BaseTransformationController. Adding a try-catch seems to work for my needs rn:

        sceneView.getScene().addOnPeekTouchListener(new Scene.OnPeekTouchListener() {
            @Override
            public void onPeekTouch(HitTestResult hitTestResult, MotionEvent motionEvent) {
                try {
                     ts.onTouch(hitTestResult, motionEvent);
                } catch (Exception ex) {
                     ex.printStackTrace();
                }
            }
        });

[ubikapps]

To get mine working I had to set the node to be selected after it had been constructed:
transformationSystem.selectNode(transformableNode)
I found a load of transformableNode.isSelected() calls in the canStartTransformation methods of the controller classes in com.google.ar.sceneform.ux package e.g. here is ScaleController's:

@Override public boolean canStartTransformation(PinchGesture gesture) { return getTransformableNode().isSelected(); }

This and the fix from @civilordergone combined worked well for me