Azure-kinect-sensor-sdk: "k4abt_tracker_create" return failed

[xjedges]

I am trying to implement latest SDK(v0.9.2) to unreal engine, since the c++ wrapper is ready in this version. The Azure-Kinect-Sensor-SDK works fine, I got all the images displayed, but "k4abt_tracker_create" keep return failed.
I don't know if I don't include all the necessary assets(.dll, .onnx) to the project since the unreal engine has "unique" way to link three-party source.
My question:
1) Is there bug track for this api?
2) Except managing the reference by "NuGet", could I import manually?

Thanks in advance

[yijiew]

@xjedges Thanks for reporting this issue. Could you try to log more verbose message to a file. There is an undocumented way to log body tracking SDK information. It is similar to the K4A log instruction https://docs.microsoft.com/en-us/azure/Kinect-dk/troubleshooting. But instead of set K4A_ENABLE_LOG_TO_A_FILE and K4A_LOG_LEVEL. Please do the following:

1. set K4ABT_ENABLE_LOG_TO_A_FILE=k4abt.log
2. set K4ABT_LOG_LEVEL=i
   It would be very helpful to debug this issue if you could post the error log here.

[Cris21395]

Hi @xjedges,

Has you solved your problem? I know this thread is not the right site for asking you this issue but, could you provide me some details about implementing the Azure Kinect SDK to unreal?

Thanks in advance.

[xjedges]

Hi @xjedges,

Has you solved your problem? I know this thread is not the right site for asking you this issue but, could you provide me some details about implementing the Azure Kinect SDK to unreal?

Thanks in advance.

I was busy with other project now, will have a test later.

[tejaswingm]

Hi @yijiew

I am also integrating Azure Kinect SDK 1.2.0 and Body Tracking SDK 0.9.3 into Unreal (4.23). But before integrating into Unreal, I made a C++ console application by following this : Build your first body tracking app. And it worked without any issues.

When I added the same code into Unreal (4.23), the k4a_device_open, k4a_device_start_cameras, k4a_device_get_calibration returned K4A_RESULT_SUCCEEDED but k4abt_tracker_create Failed.

So I followed the steps to enable logging by setting the Environment variables for both k4a and k4abt.

@xjedges Thanks for reporting this issue. Could you try to log more verbose message to a file. There is an undocumented way to log body tracking SDK information. It is similar to the K4A log instruction https://docs.microsoft.com/en-us/azure/Kinect-dk/troubleshooting. But instead of set K4A_ENABLE_LOG_TO_A_FILE and K4A_LOG_LEVEL. Please do the following:

1. set K4ABT_ENABLE_LOG_TO_A_FILE=k4abt.log
2. set K4ABT_LOG_LEVEL=i
   It would be very helpful to debug this issue if you could post the error log here.

The logs are getting created when the test C++ console application is run but logs are not created when we run our Unreal (4.23) project. So, is there any other way we could troubleshoot or debug?

@xjedges and @Cris21395, were you guys able to get logs for K4A and K4ABT from your Unreal project?

Thank you.

[tejaswingm]

Hi @yijiew

I updated the Sensor SDK to 1.3.0 and Body Tracking SDK to 0.9.4 but it did not make any difference.

One thing I found is that when running the test C++ console application in visual studio 2017 through Ctrl + F5, the k4abt_tracker_create returns FAILED and this is printed to the console :

E:\Projects\Cpp\first-azure-kinect-body-tracking\x64\Debug\first-azure-kinect-body-tracking.exe (process 23164) exited with code 1.

Although, if I directly run the .exe from the Debug folder, it works fine.

Do you guys have any idea why this would happen?

So, after I realized that, instead of running the Unreal project in the Editor (4.23), I made a build and ran the .exe and it Worked. It also created the k4a.log and k4abt.log files.

If we cannot test the Azure Kinect SDK integration into Unreal in the Editor then our workflow would become very tedious. We would have to make builds for every little change to know if its working or not.

Is there a way to avoid this?

[RoseFlunder]

Seems like you have to change your working directory in your IDE.
When you start the compiled app as executable it will find the dependency files because they are in the debug folder.
When you start it via IDE it will most likely set the working directory to your project directory or sth else.

[tejaswingm]

Seems like you have to change your working directory in your IDE.
When you start the compiled app as executable it will find the dependency files because they are in the debug folder.
When you start it via IDE it will most likely set the working directory to your project directory or sth else.

@RoseFlunder Well, you are right about the working directory. Setting the working directory to the Debug folder worked for my test C++ console application.

Now, I have to figure out how to do that for Unreal Engine.

[tejaswingm]

ok, so I put all the Azure Kinect dependent dlls and onnx files in the Unreal Engine installed folder where the UE4Editor.exe resides and Voila!, when I ran the project in the Editor and it worked. It created the tracker and displayed the number of bodies detected.

I am still trying to figure out a way to make Unreal find and link the dlls that are in another folder so that I don't need to put the dlls in the Unreal installation folder.

[Cris21395]

@tejaswingm I have followed your steps, but putting all dlls and onnx files in the UE4 installed folder doesn't work for me. Have you found another solution?

Thanks.

[tejaswingm]

@Cris21395 oh, does it give any error?

No, I haven't found another solution yet. But until then I will use this workaround to avoid the k4abt_tracker_create Failure in Unreal Editor.

[rfilkov]

@tejaswingm @Cris21395 you can debug the body tracking a bit to find out what's wrong. To do it, set these two environment variables: K4ABT_ENABLE_LOG_TO_A_FILE=k4abt.log and K4ABT_LOG_LEVEL=W. Then run the UE4 scene to get the error, close the UE Editor and finally look at what kind of error is logged in the k4abt.log-file.

The most probable reason may be that some libraries or the model file is not in the current directory. I'm not expert in UE, but there should be some API to print out the current working directory, when the scene runs. This would help you find a workaround of the issue.

[tejaswingm]

@tejaswingm @Cris21395 you can debug the body tracking a bit to find out what's wrong. To do it, set these two environment variables: K4ABT_ENABLE_LOG_TO_A_FILE=k4abt.log and K4ABT_LOG_LEVEL=W. Then run the UE4 scene to get the error, close the UE Editor and finally look at what kind of error is logged in the k4abt.log-file.

@rfilkov
I have already set the Environment variables to get the logs.

I initially thought that the logs are created in the Project folder where I put the k4a.dll and k4abt.dll. But that is not the case. The logs are created in the Unreal installation folder where UE4Editor.exe resides.

If you undo the workaround, i.e., delete the dlls that are copied into the Unreal installation folder, then k4abt_tracker_create fails when run in Editor.

Then the k4abt.log will say this :

[2019-10-22 15:22:31.240] [info] [t=23700] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (487): VerifyDependencies(). Verifying Dependencies!
[2019-10-22 15:22:31.241] [error] [t=23700] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (489): VerifyDependencies(). Cannot locate onnxruntime.dll
[2019-10-22 15:22:31.241] [error] [t=23700] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (136): Create(). At least one dependent library is missing!
[2019-10-22 15:22:31.241] [error] [t=23700] [K4ABT] D:\a\1\s\src\sdk\k4abt.cpp (38): tracker->Create(sensor_calibration, config) returned failure in k4abt_tracker_create()

Once that is resolved by copying that file to the Unreal installation folder, it complains about the other dependent files: cudnn64_7.dll, cudart64_100.dll, cublas64_100.dll, dnn_model_2_0.onnx.

That is the reason why I copied all the files into the Unreal installation folder.

---

Currently, this is my setup and as mentioned in my previous comment this is a workaround and it works.

<Project or Plugin>/Binaries/Win64/
 - k4a.dll
 - k4abt.dll
 - depthengine_2_0.dll

_Workaround solution:_

<Unreal_Installation>/Engine/Binaries/Win64/
 - onnxruntime.dll
 - cudnn64_7.dll
 - cudart64_100.dll
 - cublas64_100.dll
 - dnn_model_2_0.onnx

---

The most probable reason may be that some libraries or the model file is not in the current directory. I'm not expert in UE, but there should be some API to print out the current working directory, when the scene runs. This would help you find a workaround of the issue.

yes, I have been trying to find exactly that.

[tejaswingm]

Another thing I found is that if I add the folder path where all the dlls are, to the Environment variable's Path (user or system), then I don't need to do the workaround of copying all the dlls into the Unreal installation folder.
I tried this C:\Program Files\Azure Kinect Body Tracking SDK\tools and I also tried a custom folder where I put all the dlls and it worked both the times except for one caveat.

The dnn_model_2_0.onnx file should still be copied into the Unreal installation folder. Otherwise, it gives error since it is specifically looking for this file in the same folder as the Unreal exe.

[2019-10-22 15:52:00.344] [info] [t=12612] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (487): VerifyDependencies(). Verifying Dependencies!
[2019-10-22 15:52:00.348] [error] [t=12612] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (158): Create(). Cannot find DNN model at: E:/Program Files/Epic Games/UE_4.23/Engine/Binaries/Win64/dnn_model_2_0.onnx
[2019-10-22 15:52:00.348] [error] [t=12612] [K4ABT] D:\a\1\s\src\sdk\k4abt.cpp (38): tracker->Create(sensor_calibration, config) returned failure in k4abt_tracker_create()

Hope this helps.

[rfilkov]

@tejaswingm Congrats! I think this answers your question. The log-file gets created in the current working directory. In your case this is 'E:/Program Files/Epic Games/UE_4.23/Engine/Binaries/Win64'. The DNN model should be there, as well. As for the other DLLs, they may be in any of the folders listed in PATH. The current working directory is automatically added to the PATH, so if you put the DLLs in there, this would solve the issue without modifying the PATH variable.

[Cris21395]

Thank you @tejaswingm and @rfilkov for your answers. Adding the C:\Program Files\Azure Kinect Body Tracking SDK\tools to the Environment variable's path and placing only the dnn_model_2_0.onnx and cudnn64_7.dll into the Program Files/Epic Games/UE_4.23/Engine/Binaries/Win64 works fine for me.

[tejaswingm]

Thank you @tejaswingm and @rfilkov for your answers. Adding the C:\Program Files\Azure Kinect Body Tracking SDK\tools to the Environment variable's path and placing only the dnn_model_2_0.onnx and cudnn64_7.dll into the Program Files/Epic Games/UE_4.23/Engine/Binaries/Win64 works fine for me.

@Cris21395 you are welcome.

[qm13]

Added note to Azure Kinect troubleshooting documentation page.

[chrwel]

Hi guys. Been trying to get the camera started in UE4 by following instruction and comments above. But cant get it running. When I try to run in UE4 editor I get the following error:
AzureKinectDeviceLog: Error: Kinect device (id : 0) - Initialization Failed with error : Failed to create k4abt tracker!
AzureKinectLog: Error: Kinect device (id : 0) initialization Failed, so cleaning up.

Any thoughts?

[TobiasStaerk]

quick tip for other people struggling to get this to work: I use the most recent SDKs (Azure Kinect SKD 1.4.0 and Azure Kinect Body Tracking SKD 1.0.1) and got it working fine after I replaced the .dlls and .onnx in the \Plugins\AzureKinectUnreal\Binaries\Win64 folder with the ones installed at \Azure Kinect Body Tracking SDK\tools. Also added the environment variable etc. as recommended in original post here: https://github.com/secretlocation/azure-kinect-unreal

Maybe helpful for someone else, Cheers

[RichardNockles]

This is a very embarrassing question but... For the life of me I can't find the 'environment variable' 'Path' folder is this a windows or Unreal folder? here's the note I was following:
Add the <Azure Kinect Body Tracking SDK installation>/tools folder to the Path variable in Environment Variables for User or System.
apologies for the Noob. cheers all.

[Cris21395]

This is a very embarrassing question but... For the life of me I can't find the 'environment variable' 'Path' folder is this a windows or Unreal folder? here's the note I was following:
Add the <Azure Kinect Body Tracking SDK installation>/tools folder to the Path variable in Environment Variables for User or System.
apologies for the Noob. cheers all.

To set an environment variable type environment in the search windows box and select edit the system environment variables. Then, select Environment Variables button. After that, search for Path and add the folder Azure Kinect Body Tracking SDK installation>/tools.

I hope it helps. Happy coding!

[RichardNockles]

Thaaaaaaaankyou!!!!!!!!!

[UEBoy2019]

Thank all ;
the log
a part of log
[2020.10.21-02.11.55:618][ 0]LogSerialization: Warning: Attempting to load a BulkData object that cannot be loaded from disk
[2020.10.21-02.11.55:618][ 0]LogSerialization: Warning: Attempting to load a BulkData object that cannot be loaded from disk
[2020.10.21-02.11.55:619][ 0]LogSerialization: Warning: Attempting to load a BulkData object that cannot be loaded from disk
[2020.10.21-02.11.56:403][ 0]AzureKinectDeviceLog: Error: Kinect device (id : 0) - Initialization Failed with error : Failed to create k4abt tracker!

i have resolved it :place to Envientment variables
D:\Program Files\Azure Kinect Body Tracking SDK\tools