Azure-kinect-sensor-sdk: What is the technical difference between Azure Kinect and Kinect V2's body tracking?
After using the body tracking of Azure Kinect. We found most of the time, the Kinect V2 has better performance (speed, efficiency, and reliability) than it. So what is the technical difference between Azure Kinect and Kinect V2's body tracking except the Azure Kinect uses a DNN model to derive the skeleton, the Kinect v2 uses a ‘random forest’ model? In which situation we choose to use the Azure Kinect's body tracking instead of that of Kinect V2?
Jessica-Yeung
All 7 comments
From what I understand, the Azure Kinect's current Body Tracking SDK uses a machine learning model trained on a large data sets to extrapolate poses from the IR feed of the sensor. This approach has been quite successful for Google, Intel, WrenchAI, whereas Microsoft's attempt leaves a lot to be desired.
You can learn about the Kinect v2 Body Tracking random forest model here.
In this moment the Kinect v2 Body Tracking is superior to it's successor in pretty much all respects but I'm rooting for the developer team to get the DNN body tracking model up to speed with the other companies - it has the potential to work even better than the others because it is trained on IR rather than RGB which is superior in dark environments.
Azure Kinect was built for industrial applications, most of which rely on small form factor computers without discreet GPUs running passively all day.. so the developer team has the incentive to make body tracking work in these scenarios, but I think it'll take a while for them to get a fast/accurate/efficient DNN model up and running. Until then we're stuck with this extremely latent and slow body tracking that only runs on Nvidia gaming rigs.
Much of the developer community have put their projects on hold because of this, so I've started a feature request to implement a Legacy Body Tracking option. This would give developers the choice to to use random forest Kinect v2-style body tracking over DNN if they want - the feature request has already amassed a lot of support so hopefully we'll have a solution sooner than later.
fractalfantasy
on 11 Apr 2020
@Jessica-Yeung the team debated porting the previous Kinect for Windows body tracking algorithm to Azure Kinect vs leveraging 10+ years of research into pose estimation since we developed the original algorithm. The team opted to develop a new model built around a DNN and plan to evolve this model to improves its performance and accuracy.
qm13
on 15 Apr 2020
@fractalfantasy I think the first thing that needs to be requested is the Body Tracking SDK team to be replaced with another one, more skilled, experienced and responsive to customer requests. Just created one: https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/40188511-please-replace-the-body-tracking-sdk-team
rfilkov
on 16 Apr 2020
The current Body Tracking SDK is almost unusable on most of the devices where Kinect V2 was running smoothly. The result of this, especially in these Covid-19 times, is that we almost cannot use the new Kinect for Azure for all those applications home-based that are currently desperately needed to keep people active and in shape. Let me humbly suggest to the development team to consider opening a development branch of the previous SDK for Kinect v2 to be used with the new Kinect for Azure, of course paying the price of a less accurate tracking. Which for most of the interactive applications was already fairly good (we are ahppily using Kinect v2 + old sdk in a rehab center for children).
Other solutions (i.e. based on Intel Realsense D425 or D415), much simpler and less effective from many points of view, are already in place, and if there will be no changes in the current Kinect for Azure SDK development plans regards the use of GPU-intesive DNN for the BT, I think many will switch to those ones .
dotslinker
on 21 Apr 2020
@dotslinker I agree with you on all fronts - this topic got closed but body tracking performance is now being tracked in @rfilkov’s thread #514 so feel free to copy paste your response there if you‘d like the sdk team to see it.
Also the idea you mentioned is on the feedback forum so please upvote if you’d like to see it happen:
https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/39945454-legacy-body-tracking-like-kinect-v2
fractalfantasy
on 21 Apr 2020
@fractalfantasy thanks a lot. I did it right now. It is really unbelievable that I may reach less than 20 fps for the BT SDK on a Dell i9 32 GB Ram just because I have a low rated Nvidia GTX 1650.
And the other thing I really don't understand is that usually the newer versions of the SDKs are set on the top of the previous ones, while the new K4W SDK one seems to be rather built from scratch and definitively less usable for many aspects (i.e. source examples just to name one) if compared to the Kv2...
dotslinker
on 22 Apr 2020
Is it possible that the same virtual / ai modeling that the Azure uses for presentations could be achieved by the xbox one kinext?
Fulkeg
on 28 Jul 2020
Related issues
y0ngw00
·
3Comments
legacydev
·
3Comments
UnaNancyOwen
·
3Comments
d-walsh
·
3Comments
natelowry
·
3Comments
Most helpful comment
The current Body Tracking SDK is almost unusable on most of the devices where Kinect V2 was running smoothly. The result of this, especially in these Covid-19 times, is that we almost cannot use the new Kinect for Azure for all those applications home-based that are currently desperately needed to keep people active and in shape. Let me humbly suggest to the development team to consider opening a development branch of the previous SDK for Kinect v2 to be used with the new Kinect for Azure, of course paying the price of a less accurate tracking. Which for most of the interactive applications was already fairly good (we are ahppily using Kinect v2 + old sdk in a rehab center for children).
Other solutions (i.e. based on Intel Realsense D425 or D415), much simpler and less effective from many points of view, are already in place, and if there will be no changes in the current Kinect for Azure SDK development plans regards the use of GPU-intesive DNN for the BT, I think many will switch to those ones .