Azure-kinect-sensor-sdk: image format of body index map has been changed in v0.9.2

[UnaNancyOwen]

Describe the bug

k4a_image_format_t of k4a_image_t that returned by k4abt_frame_get_body_index_map has been changed in the Azure Kinect Body Tracking SDK v0.9.1 to v0.9.2.
It is as follows for more details.

| v0.9.1 | v0.9.2 |
|:------:|:-------:|
| K4A_IMAGE_FORMAT_CUSTOM8 | K4A_IMAGE_FORMAT_CUSTOM |

Is this a specification of Azure Kinect Body Tracking SDK, or bug?
(I like K4A_IMAGE_FORMAT_CUSTOM8.)

To Reproduce

/*
 v0.9.1 : 6 (K4A_IMAGE_FORMAT_CUSTOM8)
 v0.9.2 : 8 (K4A_IMAGE_FORMAT_CUSTOM)
*/
k4a_image_t image = k4abt_frame_get_body_index_map( frame );
k4a_image_format_t format = k4a_image_get_format( image );
std::cout << format << std::endl;

Expected behavior

If it is not an intentional change, it should be K4A_IMAGE_FORMAT_CUSTOM8.

Desktop (please complete the following information):

• OS with Version: Windows 10 (1903)
• SDK Version: Azure Kinect Sensor SDK v1.2.0, Azure Kinect Body Tracking SDK v0.9.1/v0.9.2

Additional context

[bibigone]

I would like to add the following:

1. Body depth map definitely should have K4A_IMAGE_FORMAT_CUSTOM8 because function k4a_transformation_depth_image_to_color_camera_custom doesn't support K4A_IMAGE_FORMAT_CUSTOM format.
2. K4A_IMAGE_FORMAT_CUSTOM8 and K4A_IMAGE_FORMAT_CUSTOM16 formats were introduced in version 1.2 of Sensor DK. And it was really clever decision to introduce K4A_IMAGE_FORMAT_CUSTOM8 format with the same numeric value (6) as K4A_IMAGE_FORMAT_CUSTOM format had in 1.1.x. Due to this, Body Tracking 0.9.1 returns body index maps in K4A_IMAGE_FORMAT_CUSTOM8 format when used with Sensor DK 1.2. And it was really convenient.
3. But I'm afraid that Sensor team didn't inform Body Tracking team about new formats and the way they should be used. As a result, in source codes of Body Tracking K4A_IMAGE_FORMAT_CUSTOM format is still used for body index maps. Thus, after rebuilding with updated headers from Sensor DK and publishing as 0.9.2, we now have K4A_IMAGE_FORMAT_CUSTOM as format for body index maps. oops...

[UnaNancyOwen]

@yijiew What do you think? We need a quick solution to this problem.
If it was bug, I think it is necessary to release bug fixed version (v0.9.2.1 or v0.9.3) in a near days.
Thanks,

[binxie33]

We also encounter this problem and need a fix. Please help.

[JMonkey94]

Hi All, may i know is this bug able to bring any effect on body tracking sdk performance? Possible if i use the version of 0.9.1 first while waiting for the latest version 0.9.2 issue solved? Besides I get from somewhere know that Azure Body tracking SDK does not open source. may i ask confirmation on this?

[UnaNancyOwen]

@yijiew Are there any news about this issue? Thanks,

[binxie33]

@JMonkey94 I tried downgrading to v0.9.1 but it turned out v0.9.1 lacks a function I needed: k4a_transformation_depth_image_to_color_camera_custom. So I have to wait for the bug to be fixed and my project is seriously delayed because of it. And yes Azure Body tracking SDK is close source, and I'd love to see it go open source, thus the community can help to fix this kind of minor bugs more timely...

[yijiew]

Sorry folks. I was on vacation in the past two weeks. The issue was caused exactly as @bibigone mentioned, we were not aware that the Sensor SDK introduced two additional custom image type enum. We will change the image type to K4A_IMAGE_FORMAT_CUSTOM8 in the next release. @binxie33 For anyone who needs to use k4a_transformation_depth_image_to_color_camera_custom function now, you can create your own k4a_image_t with K4A_IMAGE_FORMAT_CUSTOM8 and then copy the image content of the body map.

[binxie33]

@yijiew Thanks for the hint. k4a_transformation_depth_image_to_color_camera_custom works after creating a new k4a_image_t with K4A_IMAGE_FORMAT_CUSTOM8. So now I can settle with the workaround until the fix comes out. :)

[yijiew]

@binxie33 That is good to know! Please stay tuned for the next release :)

[yijiew]

Hi everyone, the new 0.9.3 release that solves this problem is published. You can download the latest release here: 0.9.3 | msi nuget