Openpose: Supporting depth data natively (RGB-D images, RealSense)

[marlonmisra]

Issue Summary

I was wondering if there are plans to support depth data natively? So, instead of having RGB images as input, using RGB-D images.

Type of Issue

You might select multiple topics, delete the rest:

• Enhancement / offering possible extensions / pull request / etc

[walterwang]

just curious what does depth offer?

[marlonmisra]

@walterwang my project requires a 3D skeleton of each person and their relative position. I can use Kinect and look up the depth value of each pixel. That works fine but I'd imagine that you can get even higher accuracy if your training data has a depth dimension too. Do you think that wouldn't be the case?

[gineshidalgo99]

120, #287

But in general, you can implement your own code for your custom private depth-based data following the tutorials (C++/Python). You just need to set the 3-D array of OpenPose keypoints filling the Z coordinate with your depth as others have detailed here or in #120, #287.

[themoob]

This is not a duplicate, the issue is still relevant. The linked issues are connected to using the RGB camera in Kinect, not the additional depth data layer during prediction.

[pcp16]

@themoob is right, proprietary software like MS SDK and NITE2 provide depth-based skeleton detection, but there isn't an open alternative yet.

[gineshidalgo99]

In addition, you can implement your own for your custom private depth-based data following the tutorials (C++/Python)