Sceneform-android-sdk: Customize HandMotion Animation

[malulleybovo]

I was digging through the library to find a way to change (not disable) the default hand motion animation for plane detection. I came across the .setInstructionView(View) but calling this function did not seem to do anything in my case.

In my case, I have a simple project with an ArFragment in the layout. In the activity, I perform arFragment.getPlaneDiscoveryController().setInstructionView(myCustomImageView);. This results in the same default drawable and animation showing. This happens regardless of the ImageView being in the layout or generated in code.

Am I missing steps? Or can I only set the View through the constructor?

[dsternfeld7]

At what point in the activity's lifecycle are you calling setInstructionView? It must be done in onCreate.

[malulleybovo]

All attempts done onCreate. I see this happen even in the hellosceneform sample project. I set it right after getting the arFragment reference and observe that the default animation persists.

[dsternfeld7]

Thanks, I will investigate this.

[dsternfeld7]

From testing this, I believe there is a bug here where calling setInstructionsView doesn't actually remove the default view from the view hierarchy, so you end up seeing both. We will look into addressing this. In the meantime, I think you can work around it by calling hide right before you set your own view:

arFragment.getPlaneDiscoveryController().hide();
arFragment.getPlaneDiscoveryController().setInstructionView(view);

Does that solve the problem for you?

[malulleybovo]

Interestingly, I have tried all the combinations of .show(), .hide(), and .setInstructionView(View) that made sense to me. One of which was your proposed workaround. In short, none of my attempts worked this far.

In the case above, it appears to me that .hide() will hide the default drawable and .setInstructionView(View) will not change that drawable. You just won't see that the drawable was unchanged since it is hidden. If the view used is already in the layout, you're left with the illusion that it worked. However, if you call .show() after that, what you'll end up with are both the default drawable and your view on screen even though you expected to only see your drawable.

Basically, I cannot find a way to make this work the right way since this alledged "bug" seems to interfere even with your proposed workaround. I appreciate your proposed workaround nonetheless!

[dsternfeld7]

The code I posted worked in my test, so there must be something else different about our setups.

The PlaneDiscoveryController works just by setting the visibility flag of the view. Therefore, it is expected for the instructions view to already be in the layout. However, after you call setInstructionsView, show() should only effect the view that has just been set so it sounds like something is wrong there.

No problem! Can you post your code for this? Also, what version of Sceneform are you testing on? Hopefully that will help me figure out what is different about what we're doing. Thanks!

[malulleybovo]

I got it working in the hellosceneform project. It may have been after updating to sceneform v1.3. I tried using an existing ImageView and also using a programmatically generated ImageView added to the layout. Both attempts worked. The custom image would show and the default image would stay hidden. I was also wondering: is there a way to only update the image but retain the default animation used?

[dsternfeld7]

I'm glad you got it working!

I believe you can retain the default animation by creating a HandMotionView, which inherits ImageView.

<FrameLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/sceneform_hand_layout"
    android:clipChildren="false">

  <com.google.ar.sceneform.ux.HandMotionView
      android:id="@+id/sceneform_hand_image"
      android:layout_width="match_parent"
      android:layout_height="match_parent"
      android:foregroundGravity="center"
      android:src="@drawable/my_image"
      android:scaleType="center"/>

</FrameLayout>

Internally, we will look into updating this API to make it easier to use.

[malulleybovo]

When using the layout above, I found that:

• if no code is used, this HandMotionView display on top of the default (both display on center of screen).
• if .hide() is called alone, the same as above happens.
• if .setInstructionView(View) is called alone, the same as above happens.
• if .hide() and .setInstructionView(View) are called, the default HandMotionView disappears but the custom HandMotionView gets centered at the top left corner of the screen. I tried many things to get it centered on screen again but nothing worked. It appears that this only happens when the default HandMotionView gets hidden from view.

[dsternfeld7]

Can you share your code? I'd like to see at what point in the lifecycle of the activity / fragment the functions are being called.

[malulleybovo]

The following snippets is causing the default HandMotionView to disappear but the custom HandMotionView to get centered at the top left corner of the screen. I tried many things to get it centered on screen again but nothing worked.

HelloSceneformActivity.java:

public class HelloSceneformActivity extends AppCompatActivity {
    private ArFragment arFragment;
    private View phoneImage;

    @Override
    @SuppressWarnings({"AndroidApiChecker", "FutureReturnValueIgnored"})
    // CompletableFuture requires api level 24
    // FutureReturnValueIgnored is not valid
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_ux);

        // Get AR fragment from layout
        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);

        phoneImage = findViewById(R.id.testing);

        // Disable plane discovery hand motion animation
        arFragment.getPlaneDiscoveryController().hide();
        arFragment.getPlaneDiscoveryController().setInstructionView(phoneImage);
    }
}

activity_ux.xml:

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/flContainer"
    android:clipChildren="false"
    tools:context=".HelloSceneformActivity">

  <fragment class="com.google.ar.sceneform.ux.ArFragment"
      android:id="@+id/ux_fragment"
      android:layout_width="match_parent"
      android:layout_height="match_parent" />


  <com.google.ar.sceneform.ux.HandMotionView
      android:id="@+id/testing"
      android:layout_width="match_parent"
      android:layout_height="match_parent"
      android:foregroundGravity="center"
      android:scaleType="center"
      android:src="@drawable/ic_phone"/>

</FrameLayout>

build.gradle:

buildscript {
    repositories {
        google()
        jcenter()
        mavenLocal()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:3.1.2'
        classpath 'com.google.ar.sceneform:plugin:1.3.0'
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
        mavenLocal()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

[dsternfeld7]

I was able to reproduce the issue, there is definitely a problem here thanks for pointing it out! I was able to work around the issue by doing the following:

Put the custom hand animation in it's own layout file (it must be created dynamically):

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".HelloSceneformActivity">

  <com.google.ar.sceneform.ux.HandMotionView
      android:layout_width="match_parent"
      android:layout_height="match_parent"
      android:foregroundGravity="center"
      android:scaleType="center"
      android:src="@drawable/handfoo" />

</FrameLayout>

Then in onCreate, do this:

    // Disable plane discovery hand motion animation
    arFragment.getPlaneDiscoveryController().hide();
    ViewGroup container = findViewById(R.id.sceneform_hand_layout);
    container.removeAllViews();

    // Create the new plane discovery animation and add it to the hand layout.
    phoneImage = getLayoutInflater().inflate(R.layout.hand_layout, container, true);

    // Set the instructions view in the plane discovery controller.
    arFragment.getPlaneDiscoveryController().setInstructionView(phoneImage);

[dsternfeld7]

Closing due to inactivity.

[ryan-hodgman]

Just ran into this issue again, and struggled to understand what was going on until playing around with your final suggested workaround.

The PlaneDiscoveryController class isn't doing what people are assuming it does based on the API, as it doesn't actually place the view specified in .setInstructionView(view) in the layout.

Can we add a feature request to update this API so that .setInstructionView(view) replaces the default instruction view with the specified view? Would resolve a lot of unnecessary confusion.

[michaelvogt]

@dsternfeld7

I was able to reproduce the issue, there is definitely a problem here thanks for pointing it out!

Its more than a year now that you wrote this. Any news regarding solving the problem with the instruction view? Or are you planing to leave it as it is, since there is a workaround?

If so, any tip on how to get the instruction view working inside a fragment?

[hshapley]

I agree with @ryan-hodgman. The existence of the PlaneDiscoveryController suggests that this is a scenario you care about (customizing the instructions view), but it doesn't work as expected. Poking around a bit, the salient bit of code seems to be in BaseArFragment in the onCreateView method, where it calls:

View instructionsView = loadPlaneDiscoveryView(inflater, container);
frameLayout.addView(instructionsView);
planeDiscoveryController = new PlaneDiscoveryController(instructionsView);

Specifically, that second line where it adds the view to the overall Activity FrameLayout. That bit of code is necessary for any view to show up, but calls to setInstructionView don't change the Activity FrameLayout, they just change the internal View variable. That's why the default view (the moving hand by itself) won't go away and why new views don't show up. Please fix this!

I recommend adding code in setInstructionView to remove any existing view(s) and then add the new views to the right place in the hierarchy (i.e. in the overall FrameLayout).

I'm subclassing ArFragment, and the workaround doesn't work for me. I should be able to change the instruction view in my overridden onCreateView method (without re-writing the code from BaseArFragment's onCreateView). Some other solutions that would help in my situation are to add a getInstructionView method to PlaneDiscoveryController (so that we can manually remove it from the layout) and make the planeDiscoveryController in BaseArFragment protected instead of private so that it can be set/manipulated by subclasses. That way, we could subclass PlaneDiscoveryController to get the behavior we want.