Azure-kinect-sensor-sdk: body tracking is still slow, latent & inaccurate

[t2k2]

Support for non-Nvidia GPUs + more performant body tracking was promised over 3 months ago yet we haven't seen any progress in the SDK. Can we expect this to happen before the year is over?

To Reproduce

1. Run Azure Kinect Body Tracking Viewer.
2. Stand in front of the sensor and make fast arm movements.
3. Observe the accuracy and latency of the joint positions with regard to the real arms.
4. Look at the colorized body-index map with regard to the real body.
5. Open up GPUz or any GPU monitor and check the % of GPU being used.

Expected behavior

1. 30 fps or faster
2. No perceivable latency between real world movements and body tracking.
3. Body index map to match as precisely as possible the user's silhouette on the depth frame.
4. Body tracking to be less demanding (15% of GPU or less) in means of hardware and 3rd party software requirements.

These issues weren't present in previous generations and seriously need to be addressed. If there are no sizable efforts being made to fix them it would be appropriate to open source the body tracking SDK.

[qm13]

Thank you for your feedback. We are working on both improving performance and enabling support for non-NVIDIA GPUs (Windows only). We would also very much like to get this into the hands of developers before the end of the year.

When you state "15% of GPU or less" what GPU are you considering as a benchmark?

[qm13]

Also see #514, #1016, and #1343.

[t2k2]

Would be optimal to have it run at 15% or below on i9 Intel Integrated Graphics. For AMD, maybe a RX 5300M or equivalent. With the current Nvidia limitation: GTX 1060.

Thanks for linking the topics, they are relevant but I'm asking more for an ETA... curious to know if this is being worked on full time or if the body tracking team has moved on to Hololens or some other project.

[qm13]

We have a small team actively working on the Body Tracking SDK. No ETA at the moment.