Sceneform-android-sdk: Sceneform + Augmented Images?

[arilotter]

Since sceneform handles the render loop, I don't see a way to do what the Augmented Images documentation suggests: getting the updated trackables from a Frame. Is there an alternative method that sceneform exposes to hook into the render loop and run getUpdatedTrackables(AugmentedImage.class)?

[dsternfeld7]

The ArSceneView updates the ARCore Frame object before drawing the scene. You can access the frame by calling getArFrame() from a method registered with setOnUpdateListener() which is called once per frame.

    arSceneView
        .getScene()
        .setOnUpdateListener(
            frameTime -> {
              Frame frame = arSceneView.getArFrame();
              if (frame == null) {
                return;
              }

              Collection<AugmentedImage> augmentedImages = frame
                  .getUpdatedTrackables(AugmentedImage.class);
            });

Alternatively, you can subclass Node, override onUpdate, and access the ARCore frame from there.

[arilotter]

In what point of the ArFragment lifecycle will arSceneView.getSession() not return null, so that I can run AugmentedImageDatabase.deserialize(session, inputStream) ?

[dsternfeld7]

The session becomes available when ArFragment is resumed. However, you must still check to see if it is null. It may not be available right away if camera permission still needs to be granted or ARCore needs to be installed.