Arcore-android-sdk: Playback video and sensors on emulator or ARCore test mode

[arturohernandez10]

This new kind of app really requires to be able to capture a stream of video and sensor data. That can then be replayed on the emulator. I've had odd things happen that I can't truly replicate.

I could see having a few recordings that would be used for automated regression testing.

I am sure this must have been talked about at google. If this is not possible today. What are some ideas and/or workarounds.

Otherwise please add disconnected debug support. Where AR code is not reading the Camera or sensors. But prerecorded files.

[inio]

Great idea! You've got good intuition - we're definitely looking at this.

[idstein]

Looks like this is really possible already with private ARKit APIs on Apple.

For Android we would likely need a way to record video/audio plus all raw sensor data and then stream them into ARCore on playback.
Wouldn’t a media file with meta tracks be the obvious choice? I only currently see there the limitation of having synced timestamps for video and metadata tracks.

Existing OSS recorders
Video + Sensordata in CSV https://github.com/e-lab/VideoSensors
Matroska (Video + Metatracks per sensor) https://github.com/pscholl/cmotion

[rubenhorn]

Any news on this?
It's obvious why it might not be easy to implement, since providing such a low level abstraction will probably not be trivial to add into ARCore.
Also, I don't expect any hardware manufacturers to ever include such a feature in their implementation of the HAL, but maybe this is possible through emulated hardware. (?)

[devbridie]

The Recording and Playback API allows you to play back a pre-recorded dataset which could be used for these purposes.