Depthai: Cannot run tiny-yolo-V4 on AOK-D

[luiscastro1995]

Describe the bug
After following this Colab notebook, I get the .blob, .mapping, .bin and .xml of the trained tiny-yolo-v4 which I download locally. After cloning the repo I placed the .blob in the folder depthai/resources/nn/tiny-yolo-v3 and named it tiny-yolo-v3.blob.sh14cmx14NCE1, but before I had to remove the original tiny-yolo-v3 blob model. I kept the tiny-yolo-v3.json unchanged.

In the end I run the code:

• python depthai_demo.py -cnn tiny-yolo-v3

This is the corresponding log:

(aok) luis@luis-P2440UQ:~/Projs_Software/INESC_projects/AOK/depthai$ python depthai_demo.py -cnn tiny-yolo-v3
No calibration file. Using Calibration Defaults.
Using depthai module from:  /home/luis/.local/lib/python3.7/site-packages/depthai.cpython-37m-x86_64-linux-gnu.so
Depthai version installed:  1.0.0.0
Compiled mode found: compiled for 14 shaves, 14 cmx_slices and 1 NN_engines 
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : Super/5000Mbps
Mx serial id : 14442C10B1E409D100
Loading config file
Attempting to open stream config_d2h
watchdog started 
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: valid (v5)
  Board name     : BW1098OBC
  Board rev      : R0M0E0
  HFOV L/R       : 71.86 deg
  HFOV RGB       : 68.7938 deg
  L-R   distance : 7.5 cm
  L-RGB distance : 3.75 cm
  L/R swapped    : yes
  L/R crop region: center
  Rectification Rotation R1 (left):
    0.999958,    0.002866,    0.008741,
   -0.002884,    0.999994,    0.002032,
   -0.008735,   -0.002057,    0.999960,
  Rectification Rotation R2 (right):
    0.999981,   -0.005453,   -0.002831,
    0.005447,    0.999983,   -0.002052,
    0.002842,    0.002037,    0.999994,
  Calibration intrinsic matrix M1 (left):
  854.679382,    0.000000,  646.690918,
    0.000000,  855.430969,  404.758911,
    0.000000,    0.000000,    1.000000,
  Calibration intrinsic matrix M2 (right):
  854.277344,    0.000000,  637.004456,
    0.000000,  854.839539,  403.517487,
    0.000000,    0.000000,    1.000000,
  Calibration rotation matrix R:
    0.999898,    0.008307,    0.011593,
   -0.008354,    0.999957,    0.004021,
   -0.011559,   -0.004118,    0.999925,
  Calibration translation matrix T:
   -7.479593,
    0.040785,
    0.021172,
  Calibration Distortion Coeff d1 (Left):
   -4.449849,   14.178833,   -0.001023,   -0.000998,  -12.882705,   -4.509255,   14.391972,
  -13.084028,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
  Calibration Distortion Coeff d2 (Right):
   -4.595629,   14.189158,    0.000334,   -0.002113,  -13.973203,   -4.652501,   14.404813,
  -14.179348,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
['metaout', 'previewout']
Available streams: ['meta_d2h', 'left', 'right', 'disparity', 'depth', 'metaout', 'previewout', 'jpegout', 'video', 'object_tracker', 'rectified_left', 'rectified_right', 'color']
/home/luis/Projs_Software/INESC_projects/AOK/depthai/resources/nn/tiny-yolo-v3/tiny-yolo-v3.json
depthai: Using calibration from stored in EEPROM
config_h2d json:
{"_board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":1,"NN_config":{"NN_family":"YOLO","NN_specific_metadata":{"anchor_masks":{"side13":[3,4,5],"side26":[1,2,3]},"anchors":[10,14,23,27,37,58,81,82,135,169,344,319],"classes":80,"confidence_threshold":0.5,"coordinates":4,"iou_threshold":0.5},"output_format":"detection"},"blob0_size":11813888,"blob1_size":0,"calc_dist_to_bb":true,"camera_input":"rgb","cmx_slices":14,"keep_aspect_ratio":true,"num_stages":1,"shaves":14},"app":{"enable_reconfig":true,"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":71.86000061035156,"left_to_rgb_distance_m":0.03750000149011612,"left_to_right_distance_m":0.07500000298023224,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":68.7938003540039,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":true},"camera":{"mono":{"fps":30.0,"resolution_h":400,"resolution_w":640},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":0,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->1824
size of json_config_obj that is expected to be sent to config_h2d is ->1048576
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 11813888
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 11813888 bytes to inBlob
!!! XLink write successful: inBlob (11813888)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/luis/Projs_Software/INESC_projects/AOK/depthai/resources/nn/tiny-yolo-v3/tiny-yolo-v3.blob.sh14cmx14NCE1
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
E: [global] [    799014] [python] addEvent:262  Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    799014] [python] addEventWithPerf:276   addEvent(event) method call failed with an error: 3
E: [global] [    799014] [python] XLinkReadData:156     Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream outBlob: ...
E: [global] [    799017] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    799018] [Scheduler00Thr] sendEvents:1027        Event sending failed
Closing stream outBlob: DONE.
Traceback (most recent call last):
  File "depthai_demo.py", line 418, in <module>
    dai.startLoop()
  File "depthai_demo.py", line 84, in startLoop
    p = self.device.create_pipeline(config=config)
RuntimeError: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
!!! XLink write successful: host_capture (4)
E: [global] [    799147] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -4) | event XLINK_WRITE_REQ

Stopping threads: ...
E: [xLink] [    799147] [Scheduler00Thr] sendEvents:1027        Event sending failed
Stopping threads: DONE 0.000s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
Closing all observer streams: DONE.
Reseting device: 0.
E: [global] [    799147] [Scheduler00Thr] dispatcherEventSend:53        Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [    799147] [Scheduler00Thr] sendEvents:1027        Event sending failed
Reseting: DONE.

I thought by following the tutorial I should be able to run the model right out of the box. Probably I am missing something.
Any suggestion is more than welcome.

* More details *

• Python = 3.7
• Conda Environment
• Ubuntu = 18.04

[Luxonis-Brandon]

Hi @luiscastro1995 ,

I'm not immediately sure what is going wrong here. One initial idea is around the number of classes your model has, and if that is correctly reflected in the JSON.

How many classes does it have?

Thanks,
Brandon

[szabi-luxonis]

Yolo-v4 is not supported on openvino 2020.1 (requires 2020.4+). Gen1 only supports openvino 2020.1.

Here is an example of yolo-v4 on gen2, openvino 2021.2.

[Luxonis-Brandon]

Ah awesome thanks for the catch @szabi-luxonis ! I had missed that. Recall now.

[luiscastro1995]

@szabi-luxonis thanks for the answer, it solved the problem. Since I used openvino 2020.4 to compile my yolov4-tiny I added this line to your code:

pipeline = dai.Pipeline()
## Added
pipeline.setOpenVINOVersion(dai.OpenVINO.Version.VERSION_2020_4)

and it worked fine!