Azure-kinect-sensor-sdk: Sensor not detected upon first boot on Jetson Nano

Created on 13 Jul 2020 · 9Comments · Source: microsoft/Azure-Kinect-Sensor-SDK

Describe the bug

The device fails to run on the first try on the Jetson Nano. k4aviewer fails to detect the device.
Kinect is connected to power using the attached DC adapter. Factory cables are used.
After resetting Kinect's power or rebooting Jetson, everything works correctly.

To Reproduce

Connect Kinect to the Jetson Nano (both have to be unplugged from power).
Connect Kinect to power.
Turn on Jetson Nano.
Run k4aviewer (error appears).
Unplug Kinect's power, plug in again.
Run k4aviewer.

Expected behavior

Device runs correctly every time.

Logs
[2020-07-13 14:22:42.452] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (53): k4a_device_t_create(). Created k4a_device_t 0x55946c33d0 [2020-07-13 14:22:42.453] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (29): depthmcu_t_create(). Created depthmcu_t 0x55946d99d0 [2020-07-13 14:22:42.453] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usb_cmd_priv.h (101): usbcmd_t_create(). Created usbcmd_t 0x55946d87e0 [2020-07-13 14:22:42.465] [info] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (343): find_libusb_device(). Container ID found: {cbdf6740-ba76-442e-8086-d58f4dbf9602} [2020-07-13 14:22:42.465] [info] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (224): populate_serialnumber(). Serial Number found 000875194512 [2020-07-13 14:22:42.465] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000115, PayloadSize=255 [2020-07-13 14:22:42.480] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/color_mcu/color_mcu.c (41): colormcu_t_create(). Created colormcu_t 0x55946dbb40 [2020-07-13 14:22:42.480] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usb_cmd_priv.h (101): usbcmd_t_create(). Created usbcmd_t 0x55946e17a0 [2020-07-13 14:22:42.559] [info] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (343): find_libusb_device(). Container ID found: {cbdf6740-ba76-442e-8086-d58f4dbf9602} [2020-07-13 14:22:42.559] [info] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (224): populate_serialnumber(). Serial Number found 000875194512 [2020-07-13 14:22:42.559] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/calibration/calibration.c (47): calibration_t_create(). Created calibration_t 0x55946dd660 [2020-07-13 14:22:42.559] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=10240 [2020-07-13 14:22:44.560] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (702): usb_cmd_io(). Error calling libusb_bulk_transfer for rx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:44.560] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:44.560] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:44.560] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=15360 [2020-07-13 14:22:46.561] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:46.562] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:46.563] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:46.563] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=20480 [2020-07-13 14:22:48.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:48.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:48.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:48.564] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=25600 [2020-07-13 14:22:50.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:50.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:50.564] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:50.565] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=30720 [2020-07-13 14:22:52.565] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:52.565] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:52.565] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:52.565] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=35840 [2020-07-13 14:22:54.565] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:54.566] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:54.566] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:54.566] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=40960 [2020-07-13 14:22:56.566] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:56.567] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:56.567] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:56.567] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=46080 [2020-07-13 14:22:58.569] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:22:58.569] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:22:58.570] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:22:58.570] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=51200 [2020-07-13 14:23:00.573] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:23:00.573] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:23:00.573] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:23:00.573] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (653): usb_cmd_io(). XFR: Cmd=00000111, PayloadSize=56320 [2020-07-13 14:23:02.575] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (748): usb_cmd_io(). Error calling libusb_bulk_transfer for initial tx, result:LIBUSB_ERROR_TIMEOUT [2020-07-13 14:23:02.575] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usbcommand.c (805): usb_cmd_io(usbcmd_handle, cmd, p_cmd_data, cmd_data_size, p_data, data_size, NULL, 0, bytes_read, &cmd_status) returned failure in usb_cmd_read() [2020-07-13 14:23:02.575] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (412): usb_cmd_read( depthmcu->usb_cmd, DEV_CMD_DEPTH_READ_CALIBRATION_DATA, NULL, 0, (uint8_t *)json, json_size, bytes_read) returned failure in depthmcu_get_extrinsic_calibration() [2020-07-13 14:23:02.575] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/calibration/calibration.c (47): calibration_t_destroy(). Destroyed calibration_t 0x55946dd660 [2020-07-13 14:23:02.576] [error] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (158): calibration_create(device->depthmcu, &device->calibration) returned failure in k4a_device_open() [2020-07-13 14:23:02.576] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usb_cmd_priv.h (101): usbcmd_t_destroy(). Destroyed usbcmd_t 0x55946d87e0 [2020-07-13 14:23:02.576] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/depth_mcu/depth_mcu.c (29): depthmcu_t_destroy(). Destroyed depthmcu_t 0x55946d99d0 [2020-07-13 14:23:02.577] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/usbcommand/usb_cmd_priv.h (101): usbcmd_t_destroy(). Destroyed usbcmd_t 0x55946e17a0 [2020-07-13 14:23:02.577] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/color_mcu/color_mcu.c (41): colormcu_t_destroy(). Destroyed colormcu_t 0x55946dbb40 [2020-07-13 14:23:02.578] [trace] [t=7197] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (53): k4a_device_t_destroy(). Destroyed k4a_device_t 0x55946c33d0

Desktop (please complete the following information):

OS with Version: NVida JetPack 4.4-b144 / Ubuntu 18.04.4 LTS
SDK Version: 1.4.1
Firmware version:
RGB Sensor Version: 1.6.108
Depth Sensor Version:1.6.79
Mic Array Version: 1.6.14
Sensor Config: 6109.7

Bug Firmware Investigating

Source

jmichon

Most helpful comment

It’s been months and there is no fix or any update from the Kinect team. This issue is significant and affects both the SBC we currently use in production (LattePanda Delta), and our potential replacement for it (Jetson nano). We need urgent answers to these questions:

What is the cause of this issue, is it with the Jetsons or with the K4A DKs?
Is there a solution actively been worked on and when can we expect that?
Is there a workaround we can implement in the meantime to temporary mitigate this issue?

sjdrc on 17 Nov 2020

👍2

All 9 comments

Update: this happens only when Jetson starts with the Kinect attached (and plugged into power). Everything works fine when the Kinect is connected after Jetson has been started. I'm attaching dmesg output.

dmesg.log

jmichon on 14 Jul 2020

Another update, dmesg with messages from libusb concerning only K4A. Please confirm if this is an issue with the firmware, with the particular Kinect or the problem lays somewhere in the host.

[    6.223166] usb 2-1.1.1: new SuperSpeed USB device number 4 using tegra-xusb
[    6.243593] usb 2-1.1.1: Isoc endpoint with wBytesPerInterval of 1024 in config 1 interface 1 altsetting 4 ep 129: setting to 944
[    6.257767] usb 2-1.1.1: New USB device found, idVendor=045e, idProduct=097d
[    6.257771] usb 2-1.1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    6.257775] usb 2-1.1.1: Product: Azure Kinect 4K Camera
[    6.257778] usb 2-1.1.1: Manufacturer: Microsoft
[    6.257781] usb 2-1.1.1: SerialNumber: 000875194512
[   11.147437] usb 2-1.1.1: usb_suspend_both: status 0
[   29.138843] usb 2-1.1.1: usb_suspend_both: status 0
---------- RUNNING K4ARECORDER - FAILED ----------------------------
[   96.138868] usb 2-1.1.1: usb_suspend_both: status 0
---------- KINECT DISCONNECTED FROM POWER AND RECONNECTED ----------
[  143.644097] usb 2-1.1.1: USB disconnect, device number 4
[  144.496138] usb 2-1.1.1: new SuperSpeed USB device number 7 using tegra-xusb
[  144.519070] usb 2-1.1.1: Isoc endpoint with wBytesPerInterval of 1024 in config 1 interface 1 altsetting 4 ep 129: setting to 944
[  144.533063] usb 2-1.1.1: New USB device found, idVendor=045e, idProduct=097d
[  144.533095] usb 2-1.1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  144.533119] usb 2-1.1.1: Product: Azure Kinect 4K Camera
[  144.533141] usb 2-1.1.1: Manufacturer: Microsoft
[  144.533162] usb 2-1.1.1: SerialNumber: 000875194512
[  152.011357] usb 2-1.1.1: usb_suspend_both: status 0
[  153.604426] usb 2-1.1.1: USB disconnect, device number 7
[  155.032702] usb 2-1.1.1: new SuperSpeed USB device number 9 using tegra-xusb
[  155.055920] usb 2-1.1.1: Isoc endpoint with wBytesPerInterval of 1024 in config 1 interface 1 altsetting 4 ep 129: setting to 944
[  155.070036] usb 2-1.1.1: New USB device found, idVendor=045e, idProduct=097d
[  155.070068] usb 2-1.1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  155.070092] usb 2-1.1.1: Product: Azure Kinect 4K Camera
[  155.070114] usb 2-1.1.1: Manufacturer: Microsoft
[  155.070135] usb 2-1.1.1: SerialNumber: 000875194512
[  162.987087] usb 2-1.1.1: usb_suspend_both: status 0
---------- RUNNING K4ARECORDER - SUCCESS ---------------------------
[  208.660571] usb 2-1.1.1: usbfs: process 7677 (k4arecorder) did not claim interface 0 before use
[  211.083314] usb 2-1.1.1: usb_suspend_both: status 0

jmichon on 22 Jul 2020

@qm13 @hardyzhuang any news on this?

jmichon on 3 Aug 2020

asergaz on 3 Aug 2020

❤1

Regarding the questions asked here:

Related issue: https://docs.microsoft.com/en-us/answers/questions/55354/index.html

What machine model is the host?

Vendor: ASUSTeK COMPUTER INC.
Version: 0401
Manufacturer: ASUSTeK COMPUTER INC.
Model: GR8 II
SystemType: x64-based PC

Which OS build is running on host?

OS Name: Microsoft Windows 10 Pro
OS Version: 10.0.18362 N/A Build 18362

Which usb port of host does AKinect connect to?

I think it's Port 1 & 2, but I'm not sure that's exactly the information you're looking for?

I haven't tested the restart + reset camera workaround, I'm on a different PC now and that one does not have this issue.

Sorry for the late reply, I've been on vacation.

metervara on 24 Aug 2020

👍1

We've been experiencing this exact issue with the LattePanda Delta. It's causing us significant problems. Can someone from the kinect team please give an update on this?

sjdrc on 25 Sep 2020

What is the cause of this issue, is it with the Jetsons or with the K4A DKs?
Is there a solution actively been worked on and when can we expect that?
Is there a workaround we can implement in the meantime to temporary mitigate this issue?

sjdrc on 17 Nov 2020

👍2

@qm13

sjdrc on 2 Dec 2020

It took us time to identify the root cause. Fortunately, we have had a fix in regression test now. It is a USB stack low-level fix, will take some time. Please stay tuned...thanks,