Darknet: 2-3 seconds delay on RTSP camera

[engineer1109]

I found when I detect demo on jetson TX2,
it will be 2-3 seconds delay on RTSP camera.
Who can tell me how to solve it? =.=

[AlexeyAB]

• Try to build OpenCV with GStreamer, use -DWITH_GSTREAMER=ON when you call cmake

  • https://devtalk.nvidia.com/default/topic/1003211/jetson-tx1/how-can-i-use-gstreamer-in-my-opencv-program-on-tx1-board-/1
  • https://devtalk.nvidia.com/default/topic/1001696/jetson-tx1/failed-to-open-tx1-on-board-camera/post/5117370/#5117370

• Also try to build Darknet with LIBSO=1 and run such command:

LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH ./uselib data/coco.names cfg/yolov3.cfg yolov3.weights rtsp://192.168.0.228:554

[LbinB]

It really works,but some boxes and strange lines shows,how can i fix it?

[AlexeyAB]

• Did you install OpenCV with GStreamer -DWITH_GSTREAMER=ON and it causes strange lines?

• Can you show screenshot of this some boxes and strange lines ?

[AlexeyAB]

Also comment these lines:

• https://github.com/AlexeyAB/darknet/blob/14ed6fcb6e31dd111fc5c35c31ffa6e45fe52737/src/yolo_console_dll.cpp#L401

• https://github.com/AlexeyAB/darknet/blob/14ed6fcb6e31dd111fc5c35c31ffa6e45fe52737/src/yolo_console_dll.cpp#L410

[LbinB]

Thanks a lot for your help, but i still have a problem like the picture shows, and I use a RTSP MJPEG video stream from IP camera channel 3

[AlexeyAB]

• So you can't open RTSP stream when OpenCV is compiled with GStreamer, isn't it?

• Did you remove lines to avoid some boxes and strange lines? https://github.com/AlexeyAB/darknet/issues/1893#issuecomment-437325422

[LbinB]

I remove lines and boxes by comment two lines
And I can open RTSP stream when OpenCV and even see the result,but the error shows in the terminal without show the result

[AlexeyAB]

• Un-comment this line: https://github.com/AlexeyAB/darknet/blob/14ed6fcb6e31dd111fc5c35c31ffa6e45fe52737/src/yolo_console_dll.cpp#L409

• What deleay can you get now in seconds?

[LbinB]

I find the boxes always update slowly, it still have 2-3s delay to the frames,but the frame is almost no delay to the real world

[AlexeyAB]

Your GPU on TX2 is too slow:

• Try to train yolov3-tiny.cfg model

• If you want to speedup even more, then set width=288 height=288 in cfg-file

[duohappy]

@AlexeyAB @LbinB
I am using IP Camera. Without doing anything, I read strean by RTSP, and I get 2~3 seconds delay. I try to connect with the camera producer, and they told me that I must use their SDK to get real-time stream.
So delays may be not related with darknet.

[AlexeyAB]

@duohappy

Yes, there are 3 things that adds delay:

1. Camera - usually RTSP has higher delay than native SDK

2. OpenCV - you can solve it by compiling OpenCV with -D WITH_GSTREAMER=ON and run darknet:

# for RTSP:
/darknet detector demo cfg/coco.data cfg/yolov3-tiny.cfg yolov3-tiny.weights "rtspsrc location=rtsp://192.168.1.168/main latency=30 ! decodebin ! nvvidconv ! appsink" 

# for HTTP-mjpeg:
/darknet detector demo cfg/coco.data cfg/yolov3-tiny.cfg yolov3-tiny.weights "souphttpsrc location=http://root:[email protected]:80/mjpg/video.mjpg ! decodebin ! videoconvert ! appsink"

# for Web-cam:
/darknet detector demo cfg/coco.data cfg/yolov3-tiny.cfg yolov3-tiny.weights "v4l2src ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! videoconvert ! appsink"

1. Darknet if GPU processes less frames than Camera sends, and if set #define FRAME 3
   You can solve it by compiling Darknet with LIBSO=1 and runing
   LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH ./uselib data/coco.names cfg/yolov3.cfg yolov3.weights test.mp4

[engineer1109]

@duohappy use VPU to decode video stream

[duohappy]

@duohappy use VPU to decode video stream

I use SDK to acquire real-time stream, use OPENCV to convert stream to RGB, and finally use YOLO to detect object. This is my solution. I dont know VPU until you told me, Thank you @engineer1109