Azure-kinect-sensor-sdk: Body tracking CPU processing mode not requiring cudnn

[blackccpie]

Is your feature request related to a problem? Please describe.

My problem is that when I try to run the body tracker in CPU processing mode with the following code:

...
k4abt_tracker_configuration_t tracker_config = K4ABT_TRACKER_CONFIG_DEFAULT;
tracker_config.processing_mode = k4abt_tracker_processing_mode_t::K4ABT_TRACKER_PROCESSING_MODE_CPU;
tracker = k4abt::tracker::create(calibration,tracker_config);
...

I run into an error related to the application not findind cudnn dll dependency:

2020-03-28 22:09:23.100] [error] [t=8756] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (469): VerifyDependencies(). Cannot locate cudnn64_7.dll; Please download and install cuDNN 7 at https://developer.nvidia.com/rdp/cudnn-archive
[2020-03-28 22:09:23.100] [error] [t=8756] [K4ABT] D:\a\1\s\src\TrackerHost\TrackerHost.cpp (129): Create(). At least one dependent library is missing!

I'm running on a system without cuda compatible gpu, and that's why I want to test the cpu mode, so I find it quite cumbersome that the application requests some cuda related libraries even if I tell it I want ot run on the cpu...

Describe the solution you'd like

I only want the body tracking sdk not to ask for cuda related dependencies when CPU processing mode is enabled.

Thx

[yijiew]

Thanks for filing the issue. Unfortunately, ONNX runtime dll still requires some of those CUDA dependencies even when it is running on CPU mode. Both msi package and nuget will automatically bring in those cuda dlls. Without installing CUDA drivers, have you tried to put those cuda dlls alongside your executable and run the bodytracking CPU mode? Did it work?

[blackccpie]

@yijiew : I finally made it work, as you suggested by only copying the following 3 dlls in my paths: cudnn64_7.dll, cublas64_100.dll and cudart64_100.dll. Anyway getting rid of this constraint would a nice feature.

Regards

[yijiew]

@blackccpie Thanks for giving it a try :) I agree that it would be ideal to remove these constraints. I will fire an issue to the ONNX runtime.

[yijiew]

Issue filed here: https://github.com/microsoft/onnxruntime/issues/3537

[qm13]

ONNX runtime requires cudart64_7.dll (not the other two dlls). We plan to change the dependency check for CPU mode to checking for cudart64_7 only and work with ONNX team to fix the runtime dependencies.

[Cris21395]

@yijiew : I finally made it work, as you suggested by only copying the following 3 dlls in my paths: cudnn64_7.dll, cublas64_100.dll and cudart64_100.dll. Anyway getting rid of this constraint would a nice feature.

Regards

Hey @blackccpie, I don't know if I am clear. Have you achieved running body tracking sdk on windows system without graphic card? Thx.

[blackccpie]

@Cris21395 : yes, I do succeeded in running the body tracking sdk on a machine without NVIDIA gpu,provided I have the 3 aforementioned dlls in my path. The cpu runtime is very slow, but it works. Hope it helps.

[diablodale]

Related to ONNX runtime and CUDA DLLs...

From my diagnostics, the onnxruntime.dll does not manage its own DLL dependencies. For example, onnxruntime.dll does not look in its own directory for its needed files like cudart64_100.dll. I isolated this issue using sysinternals Process Monitor and watched onnxruntime.dll do work, and then it (or the OS) search about 40 locations for the file cudart64_100.dll and never find it. And therefore the app fails.

I inspected those 40 locations and the first place it looked was the directory of the initial root EXE that triggered all other code loading. ONNX never looked in the directory of itself. As can be seen in my scenario and Unity https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1147 this is not wise. We no longer build monolithic solutions/apps with a single EXE with all needed files in one directory.

Instead, I suggest onnxruntime.dll manage its own dependencies. If it needs to load 1-3 CUDA dlls, then I recommend it manage that itself by using a standard installer/setup and/or hook the DLL loading mechanism or the DLL path SetDllDirectoryW() to manage itself.

If this is not done, then hacks like the above Unity issue, or code I'm now writing has to micromanage the "blackbox" DLL loading behavior of onxruntime.dll. Micromanaging is never good and ripe for failure due to unknown onxruntime.dll changes that can occur at any time in "the blackbox".

I recommend as the ONNX runtime and its dependent DLLs are sorted, that it includes the self-management of its own dependent DLLs.

[qm13]

Thanks for the recommendation @diablodale. I have passed it on to the ONNX team.