Sceneform-android-sdk: [Feature Request] Video recording

[Deishelon]

Hello! I'd love to request video recording feature.
I know that it's possible to do via OpenGL's add-on https://github.com/UncorkedStudios/recordablesurfaceview

Or is there any way to inject this to use with Sceneform?

[malik-at-work]

We don't currently have this as a feature. In the past I've used external applications to do recording.

I've added this feature request for consideration.

[romainguy]

You can probably just grab the Surface from the SurfaceView used by Sceneform and send it to the MediaEncoder to record a video. We should make this easier in our APIs though.

[oec08]

@romainguy Would you happen to have an example of how to do this with the current implementation?

[romainguy]

@oec08 Take a look at the grafika samples, they may have what you're looking for: https://github.com/google/grafika

[phoenixxt]

@romainguy I can't find SurfaceView recording shows in those samples. Could you, please, point me a little in the right direction?

[stevesamknows]

@phoenixxt Suggest having a look at
"Android Breakout game recorder patch"
on page
https://www.bigflake.com/mediacodec/

(That should be the basics for recording a GL surface like the SceneForm SurfaceView)

Caveat: if it's feasible to add a fullscreen toggle (so your AR view fills the entire screen), and

• that aspect ratio would be acceptable as final video format
• you don't need total control over in-app video recording

a 3rd party recorder is definitely the easiest solution

[phoenixxt]

@romainguy I looked at the grafika samples and tried to understand them, but the main part for them is rendering everything twice and I'm not sure I can get that lvl of access to Sceneform renderer.

[niusounds]

@romainguy

You can probably just grab the Surface from the SurfaceView used by Sceneform and send it to the MediaEncoder to record a video. We should make this easier in our APIs though.

I try this but it doesn't work. Error message in logcat is

java.lang.IllegalArgumentException: not a PersistentSurface
    at android.media.MediaRecorder.setInputSurface(MediaRecorder.java:165)
    ...

Here is my code.

MediaRecorder mediaRecorder = new MediaRecorder();
mediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
mediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mediaRecorder.setOutputFile(targetFile.getAbsolutePath());
mediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
mediaRecorder.setVideoSize(arFragment.getArSceneView().getWidth(), arFragment.getArSceneView().getHeight());
mediaRecorder.setVideoFrameRate(30);
mediaRecorder.setVideoEncodingBitRate(10000000);
mediaRecorder.setInputSurface(arFragment.getArSceneView().getHolder().getSurface()); // <- Above exception is thrown here
try {
  mediaRecorder.prepare();
  mediaRecorder.start();
} catch (IOException e) {
  e.printStackTrace();
}

[phoenixxt]

@niusounds it won't work that way, because you can only pass a persistent input surface that was created by MediaCodec api, for example.

[Deishelon]

You can probably just grab the Surface from the SurfaceView used by Sceneform and send it to the MediaEncoder to record a video. We should make this easier in our APIs though.

AR Core & Sceneform are now 1.5.0, any improvements to grab the Surface?
Thanks! Any samples will be highly appreciated

[Kanunnikoff]

Good afternoon.
It would be great to have an ability to record AR video. If it is possible, please do it.
Thank you very much in advance.
Regards.

[siranjeevi-aximsoft]

I followed RecordFBOActivity from https://github.com/google/grafika

Via Choreographer.getInstance().postFrameCallback(this);

I just try grab window surface and GL state. No use.. Its recording black screen only..

Anyone have idea ..would be great.. Any samples will be highly appreciated

[w2xx]

@malik-at-work I'm glad scene v1.6 added video recording, but what if I just wanted to record the original data of the camera, not the data rendered by arcore?

[siranjeevi-aximsoft]

Thank's a lot w2xx. Will check and let you know the feedback's

[Kanunnikoff]

Thank you very much for your work, guys.

[claywilkinson]

The videorecording sample (added in 1.6) demonstrates how to record the SceneView.

[w2xx]

@claywilkinson what if I just wanted to record the original data of the camera, not the data rendered by arcore?

[siranjeevi-aximsoft]

@w2xx.
I have too struggling with same issue.
Mirroring sceneview using MediaRecorder.getSurface() records entire surface view (with AR objects)
@claywilkinson
There is any Feature to record video with original camera data?
Any samples will be highly appreciated.

[w2xx]

I apologize. Because at present we only want to do some guiding functions through ar, but I do need this function here, and I find that arkit can do it.

[claywilkinson]

For something like that you'd need to stick to the low level OpenGL implementation of ARCore. This way you can get access to the texture that is the camera image and then copy it to the media recorder or whatever you want to do.

Sceneform is intended to make common 3d graphics functionality easy when developing for AR on Android.

Best of luck!

[w2xx]

@claywilkinson thanks