Azure-kinect-sensor-sdk: Body orientation flip when wearing glasses

Created on 2 Aug 2019 · 16Comments · Source: microsoft/Azure-Kinect-Sensor-SDK

Hello, we have an issue with the body tracker when we wearing an hood and protection glasses (see image below). In these case the body constantly flip.
I thing this come from the difficulty of the body tracker to well measure the eyes and/or the ears.
As you can see on the enclosed video, the skeleton id change often and the skeleton is flipped to 180°. This phenomenon was only found for some people of my team, not for all people.
This is a huge issue in industrial or health applications because most people have such protection equipment or hygienic cloths.

To Reproduce

Tested in Simple_3D_Viewer sample
Wear protection glasses or other

Expected behavior
The global orientation stability need to be improved.
Some ideas :

Detecting flip and correct it.
- Take another body part to compute the global orientation.
- Improve the eyes and ears tracking.

Screenshots
testBTGlass

Desktop (please complete the following information):

OS with Version: OS with Version: Windows 10
SDK Version: 1.0.1

Body Tracking Bug Triage Approved

Source

PierrePlantard

Most helpful comment

This issue is a big one for me as well, I am not able to adopt this camera at this time because of the body flipping. It doesn't seem to be just isolated to the glasses, (please note issue #1074 ). I have had this issue often while trying to implement this camera in my current project. Could we please prioritize this issue? It is the main reason I can't adopt the Azure Kinect DK camera in my current product (and I would so love to switch over to this).

For the record I am using SDK 1.3.0, tracker 1.0.0, 'K4A_DEPTH_MODE_NFOV_UNBINNED', 'K4A_IMAGE_FORMAT_COLOR_BGRA32', 'K4A_COLOR_RESOLUTION_720P'

Might I ask (without diving too far into the codebase)... how are we currently determining the orientation of the body in body-tracking mode?

Chrisbright10 on 8 Mar 2020

👍4

All 16 comments

I've seen these kind of flips (as well as the tracking being lost and being picked up as another body ID as can bee seen in your clip) on occasions when not wearing glasses too.

It seems fairly robust on simple poses standing and walking but when crouching, bending, rotating it can be quite fragile. The 360 degree rotation tracking is nice though, as is the quick identification.

Brekel on 2 Aug 2019

Thank you for reporting this issue. We are investigating fixes to make flipping less common.

cdedmonds on 6 Aug 2019

Thank you for your response.
Keep us in touch.

Best regards,

Pierre

Le mar. 6 août 2019 à 19:23, Chris Edmonds notifications@github.com a
écrit :

Thank you for reporting this issue. We are investigating fixes to make
flipping less common.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/575?email_source=notifications&email_token=AALQ6V5TYDJJLBINAXMBPCDQDGXQPA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3V3SNQ#issuecomment-518764854,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALQ6VZMW3OSIDU47V3T2ITQDGXQPANCNFSM4II4KJ6Q
.

PierrePlantard on 6 Aug 2019

Hi, we've been testing the Azure Kinect for a while now and it seems the biggest issue we're facing is this random flipping (SDK 1.2.0, tracker 0.9.2, 'K4A_DEPTH_MODE_NFOV_UNBINNED', 'K4A_IMAGE_FORMAT_COLOR_BGRA32', 'K4A_COLOR_RESOLUTION_720P').

Due to our use case we either have to mount low (~floor height) or high (~2.3m), tilted up/down to track a person standing ~2.5m away. Our preference is low as it seems to provide better tracking of smaller people/children and fewer issues with perspective/lean.

The problem we're seeing is that in the low position this flip issue is occurring quite frequently, often when just standing still (without glasses). It seems leaning slightly towards the camera (showing it more face) helps but is not a practical fix. In the high position this flipping is far less prevalent, I'd presume down to more face being visible, but instead opens up other issues such as not tracking shorter people and more acute angles to compensate for.

I'm currently looking into trying to catch the flip somehow and filter it out but wondered if you could provide any updates on this issue or any further guidance on how to minimise it?

Cheers,
Richard

RenderHeadsMrT on 16 Sep 2019

👍1

Hi,
Any update relative to this bug ?
It is also the biggest issue we are facing.
I can send you some example in video to illustrate the problem (in private).
Keep us in touch,

Cheers,
Pierre

PierrePlantard on 21 Nov 2019

N^ Hot. Pick wisely please.

On Nov 21, 2019, at 8:24 AM, "PierrePlantard" <[email protected]notifications@github.com> wrote:

Hi,
Any update relative to this bug ?
It is also the biggest issue we are facing.
I can send you some example in video to illustrate the problem (in private).
Keep us in touch,

Cheers,
Pierre

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FAzure-Kinect-Sensor-SDK%2Fissues%2F575%3Femail_source%3Dnotifications%26email_token%3DAL7WBXEHAEBUDA3CV6TYLCDQU2Y3DA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2ZZLQ%23issuecomment-557161646&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602927450&sdata=CqgkceOYm2dZfcj8Ck%2FtDCzcWNGaFJj7tyOo6TPd4%2Bg%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAL7WBXFSU4DNN6OGVANQ4IDQU2Y3DANCNFSM4II4KJ6Q&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602937461&sdata=jwmXLrt1Se12eUAClYwMt00rIS2SXw4Yx2jKH5yWU6U%3D&reserved=0.

jesd23 on 21 Nov 2019

Blu easy to near grn two not one malt .

On Nov 21, 2019, at 8:31 AM, "James Edmond Smith" notifications@github.com> wrote:

N^ Hot. Pick wisely please.

On Nov 21, 2019, at 8:24 AM, "PierrePlantard" notifications@github.comnotifications@github.com> wrote:

Hi,
Any update relative to this bug ?
It is also the biggest issue we are facing.
I can send you some example in video to illustrate the problem (in private).
Keep us in touch,

Cheers,
Pierre

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FAzure-Kinect-Sensor-SDK%2Fissues%2F575%3Femail_source%3Dnotifications%26email_token%3DAL7WBXANYHRHVDZN4UTL2GLQU2ZWRA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE22UXI%23issuecomment-557165149&data=02%7C01%7C%7C3d56a4b3bf024cbad20e08d76ea047aa%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099506979875861&sdata=UdT8rR1ArtJdzcDjGkZb06kx%2FjrRE9pVqBdLDZxLBkU%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAL7WBXFKQJ742VMCY7REAODQU2ZWRANCNFSM4II4KJ6Q&data=02%7C01%7C%7C3d56a4b3bf024cbad20e08d76ea047aa%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099506979885871&sdata=Qc4v1mSS5FfDDriVx1FBPoyw9sm%2FQMHTzDu8wtGw3bE%3D&reserved=0.

jesd23 on 21 Nov 2019

Could you please send us a recording illustrating the issue.

qm13 on 22 Nov 2019

Thank you,
do you have an email to send you the video ?
Pierre

PierrePlantard on 22 Nov 2019

Dear @qm13, please give me an email to send the video.

Regards,
Pierre

PierrePlantard on 25 Nov 2019

@PierrePlantard we would like to get a recording that demonstrates the problems you are reporting. We require a signed data release agreement along with the recording. Could you provide me with contact details so I can provide you both the agreement and upload instructions?

qm13 on 3 Dec 2019

@qm13 yes, you can email me at pierre.[email protected]
Thanks

PierrePlantard on 4 Dec 2019

Dear @qm13, I still haven't received your email, is it normal ?

PierrePlantard on 11 Dec 2019

For the record I am using SDK 1.3.0, tracker 1.0.0, 'K4A_DEPTH_MODE_NFOV_UNBINNED', 'K4A_IMAGE_FORMAT_COLOR_BGRA32', 'K4A_COLOR_RESOLUTION_720P'

Might I ask (without diving too far into the codebase)... how are we currently determining the orientation of the body in body-tracking mode?

Chrisbright10 on 8 Mar 2020

👍4

Is there any update on this issue? If not, who can I pay to help fix this issue faster? This is the only issue preventing me from adopting this camera at this time, and I would really love to.

Chrisbright10 on 4 Jun 2020

I have exactly the same problem - when I wear glasses the problem occurs very often (every several seconds).