Azure-kinect-sensor-sdk: Offset in Skeletons from Azure Kinect and Kinect v2

[justamad]

Hello,

for my use case, I want to register the Skeletons from Azure Kinect to the Skeletons coming from Kinect v2. In the calibration process, I'm using an 8x5 50mm checkerboard and identify three corner points using OpenCV, and for each Kinect, using the mapper functions to go from 2D (x,y) color pixel location to 3D (x,y,z) camera space (see image 1,2)

After moving the checkerboard 4-6 times and capture new calibration images for both cameras, I'm using SVD to get a rotation and translation between the two camera locations. The mean squared error of the rotated points is usually less than 1cm. However, when applying the resulting rotation+translation onto the skeletons, I receive an offset in the y-axis, when x and z-axis are looking good, as shown in the image.

System Setup Azure:
OS: Windows 10
GPU: Nvidia RTX 2060
RAM: 16GB
Azure Kinect SDK: 1.3.0
Body Tracking: 1.0.0

Kinect v2:
OS: Windows 10
Library: PyKinect2 (Python) (with Kinect Studio)

[davidkitman]

Could this be related to the tilt of the depth camera on the Kinect Azure vs its RGB camera?
"The depth camera is tilted 6 degrees downwards of the color camera"
Are you transforming for that?
https://docs.microsoft.com/en-us/azure/kinect-dk/coordinate-systems

[justamad]

Hi davidkitman,

thanks for your advice! I manually corrected for it by applying a rotation matrix of 6 degrees around the x-axis and now the result is looking much better, see the image below.