Edgetpu: Jetson Nano M.2 - Failed to load delegate from libedgetpu.so.1

[rdejana]

Close to having this working, but the unable to load the library.

python3 classify_image.py \

--model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels models/inat_bird_labels.txt \
--input images/parrot.jpg
Traceback (most recent call last):
File "/home/rdejana/.local/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 161, in load_delegate
delegate = Delegate(library, options)
File "/home/rdejana/.local/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 120, in __init__
raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "classify_image.py", line 122, in
main()
File "classify_image.py", line 99, in main
interpreter = make_interpreter(args.model)
File "classify_image.py", line 73, in make_interpreter
{'device': device[0]} if device else {})
File "/home/rdejana/.local/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 164, in load_delegate
library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1

lsmod:
lsmod
Module Size Used by
zram 26166 4
overlay 48691 0
spidev 13282 0
apex 17970 0
gasket 118068 1 apex
nvgpu 1575721 20
bluedroid_pm 13912 0
ip_tables 19441 0
x_tables 28951 1 ip_tables

ls /dev/apex_0
/dev/apex_0

odinfo gasket
filename: /lib/modules/4.9.140-tegra/updates/dkms/gasket.ko
author: Rob Springer [e-mail]
license: GPL v2
version: 1.1.3
description: Google Gasket driver framework
srcversion: 069B6D0F6AE12073F4EAF5D
depends:
vermagic: 4.9.140-tegra SMP preempt mod_unload modversions aarch64
parm: dma_bit_mask:int

modinfo apex
filename: /lib/modules/4.9.140-tegra/updates/dkms/apex.ko
author: John Joseph [e-mail]
license: GPL v2
version: 1.1
description: Google Apex driver
srcversion: 508A8A34D57322CEA287D17
alias: pci:v00001AC1d0000089Asvsdbcsci*
depends: gasket
vermagic: 4.9.140-tegra SMP preempt mod_unload modversions aarch64
parm: allow_power_save:int
parm: allow_sw_clock_gating:int
parm: allow_hw_clock_gating:int
parm: bypass_top_level:int
parm: trip_point0_temp:int
parm: trip_point1_temp:int
parm: trip_point2_temp:int
parm: hw_temp_warn1:int
parm: hw_temp_warn2:int
parm: hw_temp_warn1_en:bool
parm: hw_temp_warn2_en:bool
parm: temp_poll_interval:int

dmesg | grep apex
[ 5.354227] apex 0000:01:00.0: can't enable device: BAR 0 [mem 0x00000000-0x00003fff 64bit pref] not claimed
[ 5.365610] apex 0000:01:00.0: BAR 2: assigned [mem 0x20000000-0x200fffff 64bit pref]
[ 5.365748] apex 0000:01:00.0: BAR 0: assigned [mem 0x20100000-0x20103fff 64bit pref]
[ 5.365936] apex 0000:01:00.0: enabling device (0000 -> 0002)
[ 10.614865] apex 0000:01:00.0: Apex performance not throttled due to temperature
[ 128.375802] apex 0000:01:00.0: Apex performance not throttled due to temperature
[ 235.893286] apex 0000:01:00.0: Apex performance not throttled due to temperature

lspci | grep 089a
01:00.0 System peripheral: Device 1ac1:089a

dmesg | grep gasket
[ 5.332185] gasket: loading out-of-tree module taints kernel.

thanks

[Namburger]

@rdejana Could you share the output of uname -a? If the kernel is newer than 4.15, it's possible that the default apex/gasket driver is still being loaded after installing our drivers.
Please also share dmesg | grep pcie
thanks!

[rdejana]

Hi @Namburger, here you go

Linux nano 4.9.140-tegra #1 SMP PREEMPT Mon Dec 9 22:47:42 PST 2019 aarch64 aarch64 aarch64 GNU/Linux

[ 0.984258] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.3, lane = pcie-0, function = pcie-x1
[ 0.984348] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.4, lane = pcie-1, function = pcie-x4
[ 0.984439] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.5, lane = pcie-2, function = pcie-x4
[ 0.984524] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.6, lane = pcie-3, function = pcie-x4
[ 0.984611] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.7, lane = pcie-4, function = pcie-x4
[ 0.984696] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.8, lane = pcie-5, function = xusb
[ 0.984781] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.9, lane = pcie-6, function = xusb
[ 0.994476] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[ 0.995788] tegra-pcie 1003000.pcie: PCIE: Enable power rails
[ 0.996158] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
[ 1.000171] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
[ 1.044967] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
[ 1.067712] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[ 1.067724] pcie_pme 0000:00:01.0:pcie001: service driver pcie_pme loaded
[ 1.067933] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[ 1.067943] pcie_pme 0000:00:02.0:pcie001: service driver pcie_pme loaded

[rdejana]

And this might be interesting (first time I've seen in logs)

dmesg | grep apex
4.528225] apex 0000:01:00.0: can't enable device: BAR 0 [mem 0x00000000-0x00003fff 64bit pref] not claimed
[ 4.540316] apex 0000:01:00.0: BAR 2: assigned [mem 0x20000000-0x200fffff 64bit pref]
[ 4.540444] apex 0000:01:00.0: BAR 0: assigned [mem 0x20100000-0x20103fff 64bit pref]
[ 4.564046] apex 0000:01:00.0: BAR 0: error updating (0x2010000c != 0xffffffff)
[ 4.593525] apex 0000:01:00.0: BAR 0: error updating (high 0x000000 != 0xffffffff)
[ 4.675875] apex 0000:01:00.0: Refused to change power state, currently in D3
[ 4.818330] apex 0000:01:00.0: enabling device (0000 -> 0002)
[ 4.820543] apex 0000:01:00.0: Couldn't initialize interrupts: -22
[ 9.847318] apex 0000:01:00.0: Apex performance not throttled due to temperature

[rdejana]

Think I have another issue. Noticed that once I added the PCIe driver, my wired network connection has become very unstable

[Namburger]

@rdejana Hi, although I haven't seen these issue before, we have had success stories from jetson nano users. There is one thing that you can try:
Could you add pcie_aspm=off to your kernel command arguments?
You can do with by opening this file:

$ cat /boot/extlinux/extlinux.conf
TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet

Change "APPEND ${cbootargs} quiet" to "APPEND ${cbootargs} quiet pcie_aspm=off"
Then reboot your board, let me know if this fixes your issue!

[rdejana]

So far so good!
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
14.0ms
2.7ms
2.5ms
2.6ms
2.6ms

Going to keep an eye on things and close this one out!
Thanks @Namburger !