Azure-kinect-sensor-sdk: Document using headless Linux with Azure Kinect Sensor SDK

[ruis1996]

I want to use the terminal to store the depth map remotely, but sdk seems to have to use the desktop environment to open the depth camera. Is there any solution?

[RoseFlunder]

You could maybe try to setup an x-server-dummy like this:

https://techoverflow.net/2019/02/23/how-to-run-x-server-using-xserver-xorg-video-dummy-driver-on-ubuntu/

[rabbitdaxi]

@ruis1996 thank you for the feedback, may I ask, the machine that you want to remote in (the one connect to the device), are talking about linux or windows?

• if it is linux, you can try what @RoseFlunder suggested, and please take a look this issue #562
• if it is windows, you should be able to do that without problem

[ruis1996]

@rabbitdaxi @RoseFlunder
Thank you for your reply.
My sys is Ubuntu18.04.3-mate.
I will try the methods you provided, I will return feedback later, thank you.

[ruis1996]

@rabbitdaxi @RoseFlunder
I tried @rabbitdaxi ‘s method, but it can't work in my system. Is there any other way to solve it?

[rabbitdaxi]

@ruis1996 , can you also try "export DISPLAY=:1" or "export DISPLAY=:0" not sure whether it is helping in your case, but it seems help me with a putty remote to a Linux machine through SSH. (And please also ensure you followed the page Linux Device setup to run SSH without need to use sudo)

Here is what I tried:
On Linux machine (the machine that connected to a device and you are trying to remote into)

• Get the Ip address from the Linux machine. (ifconfig or ip address)
• Run sudo apt install openssh-server on the Linux machine
• Copy 'scripts/99-k4a.rules' into '/etc/udev/rules.d/'.
• Detach and reattach Azure Kinect devices if attached during this process.

On your machine (in my case it is a windows machine)

• Install putty on your current machine
• Open Putty to enter the ip address with port 22
• In the Putty SSH window, sdk/build/bin folder:
  
  
  
  • run ./depth_ft --gtest_filter=30NFOV_UNBINNED, you should see DE return error code 204
  
  
  • run export DISPLAY=:1
  
  
  • run ./depth_ft --gtest_filter=30NFOV_UNBINNED again, you should see the test passed, if it failed, try “export DISPLAY=:0” and run the test again
  
  

[RoseFlunder]

@rabbitdaxi
Your method works me when I have a monitor connected to the remote PC to which I connect via SSH.

When I disconnect the monitor and restart the remote pc and connect via SSH again, exporting a DISPLAY environment variable is not sufficent because the X-Server isn't running and no graphics driver is loaded as far as I can see it.

I am searching for a solution to use the sdk (sensor & body tracking) on a headless (no monitor connected) ubuntu machine.

For the moment connecting via SSH and starting the application would be sufficent.
In the future the app should run as a systemd service that will be started automatically as a service when the machine starts.

[StevenButner]

@rabbitdaxi ... I have been trying to reproduce the successful attempt to run the depth_ft tests in a headless mode as reported by @RoseFlunder (which he says works only when a monitor is attached). On my Ubuntu 18.04 system, I cannot get it to work with or without a monitor. I have carefully followed the instructions about installing the udev rules and also the creation of the dummy X server (via the dummy-1920x1080.conf file). My system works a lot like the instructions referenced in the post of @RoseFlunder on Sep 25 except I can see the message below in the /var/log/Xorg.0.log.

[ 933.652] (II) AIGLX: Screen 0 is not DRI2 capable
[ 933.652] (EE) AIGLX: reverting to software rendering

I am of the opinion that the DRI2 capability is required. Can anyone confirm this?

When I run the depth_ft tests, I get the DE 204 failures every time the K4A cameras attempt to start streaming.

I also see the following lines in my Xorg logs .... I wonder if these may be helpful in figuring what is wrong.
[ 1161.649] (**) Azure Kinect 4K Camera: always reports core events
[ 1161.649] (**) Option "Device" "/dev/input/event8"
[ 1161.649] (**) Option "_source" "server/udev"
[ 1161.650] (II) event8 - Azure Kinect 4K Camera: is tagged by udev as: Keyboard
[ 1161.650] (II) event8 - Azure Kinect 4K Camera: device is a keyboard
[ 1161.650] (II) event8 - Azure Kinect 4K Camera: device removed
[ 1161.692] (**) Option "config_info" "udev:/sys/devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3.1/2-3.1:1.0/input/input40/event8"
[ 1161.692] (**) Option "xkb_model" "pc105"
[ 1161.692] (**) Option "xkb_layout" "us"
[ 1161.692] (WW) Option "xkb_variant" requires a string value
[ 1161.692] (WW) Option "xkb_options" requires a string value
[ 1161.716] (EE) xf86OpenSerial: Cannot open device /dev/input/event8 No such device.
[ 1161.716] (II) event8: opening input device '/dev/input/event8' failed (No such device).
[ 1161.716] (II) event8 - failed to create input device '/dev/input/event8'.
[ 1161.716] [dix] couldn't enable device 13
[ 1161.716] (EE) Couldn't init device "Azure Kinect 4K Camera"
[ 1161.716] (II) UnloadModule: "libinput"

I realize that the depth computations performed via OpenGL are considered proprietary and closed-source. But, if users cannot run their systems without an Xserver and a monitor then almost any headless application (e.g. in the field of autonomous robotics) will not be able to use this otherwise-excellent depth-sensing camera. It is imperative that the full and exact set of pre-conditions and run-time requirements be written down. It simply isn't correct to claim full compatibility across Windows and Linux systems unless the rest of the environmental assumptions are clarified.

My application of the K4A's uses two cameras per system to acquire images with depth for use in navigation and obstacle avoidance on an autonomous robotic platform. In order to utilize the K4A, we must find a way to reliably attach the cameras and run them within a Linux-hosted system (currently Ubuntu-18.04). Ideally, we would launch the depth acquisition application process at boot time via systemd (i.e. as a daemon). But discovering and adapting to the combination of the largely-undocumented requirements regarding the K4A's OpenGL environment, together with the requisite systemd user login and Xserver configuration needs for headless systems is daunting in complexity. A complete example is needed.

[sjdrc]

Can someone from Microsoft please give an update on this issue?

There are various bits of information scattered over a few issues, but there is no official or simple confirmed working solution for using a kinect from a headless linux environment, and myself and others have tried workarounds suggested but with no success. This use case is very common for research purposes.

[palavrov]

I would like to join @sjdrc request - we need k4a working on headless linux environment - ideally on nVidia Jetson i.e. right now depth plugin is only for x86 and requires display and opengl 4.4 which completely prevent us to use the scanner in our products.

[sjdrc]

Why has this been closed? There is no documentation on how to use the Kinect in a headless environment.

[RoseFlunder]

Just to give some information about how I am working at the moment with my headless Ubuntu PC:
I am using automatic login at Ubuntu (which will start a X-Server Session and the NVidia driver starts too) together with a ssh server.
I haven't tried to autostart any app yet, but when I connect via SSH after the ubuntu PC has started, I just have to set the DISPLAY env variable with:
export DISPLAY=:0
Afterwards I can start the app that is using the kinect sensor sdk together with the body tracking sdk without an error.

So the only the thing that is missing for you would be way to start up an app after the auto login is complete.
Previously I tried my luck with systemd service, but I wasn't successfull with that.
Guess it is not designed to start applications that need a X-Server session + running NVidia Driver.

Hope this helps some of you.

[wes-b]

Thanks @RoseFlunder for the update.

This is closed because issue is a duplicate of #810.