Darknet: error: ‘Stream’ has not been declared

[xhuvom]

I am trying to compile this fork on Ubuntu 16 with the following settings:
GPU=1 CUDNN=1 OPENCV=1 DEBUG=0 OPENMP=0 LIBSO=1

However, the compilation returns the following error:
In file included from /usr/local/include/opencv2/cudaoptflow.hpp:50:0, from src/yolo_v2_class.hpp:12, from src/yolo_v2_class.cpp:1: /usr/local/include/opencv2/core/cuda.hpp:158:33: error: ‘Stream’ has not been declared void upload(InputArray arr, Stream& stream); ^ compilation terminated due to -Wfatal-errors. Makefile:77: recipe for target 'darknet.so' failed make: *** [darknet.so] Error 1

[AlexeyAB]

@xhuvom What version of OpenCV do you use?

[xhuvom]

I am using OpenCV version 2.4.13 and CUDA 8.0

[AlexeyAB]

Did you uncomment this line before compiling? https://github.com/AlexeyAB/darknet/blob/ea09a6e0b38e1ddf43ffcd81d27f0506411eb8e4/src/yolo_console_dll.cpp#L16

Tracking supports now only OpenCV 3.x, and this part doesn't support OpenCV 2.x
So you should change this code for OpenCV 2.x if you want to use Tracking using optical flow.

[xhuvom]

I have tried to uncomment the line and recompile but no success. The error still persists. For a better clarification, shouldn't I comment out the line as I am using OpenCV 2.4.13 ? If I would use OpenCV 3.X then this line should work, am I right?

[AlexeyAB]

@xhuvom I fixed it. Just update code from this repo, leave //#define TRACK_OPTFLOW commented, and recompile.

Yes, it should work with commented line on OpenCV 2.4.13.
And should work with commented/uncommented line on OpenCV 3.x.

[xhuvom]

Thanks @AlexeyAB for the update. Meanwhile I have updated to OpenCV 3.4 and its working fine. However, the object tracking isn't robust yet. Any suggestion on tweaking the tracking feature for better performance?

[AlexeyAB]

• What GPU do you use?
• How many FPS shown for Detection and Capture at the top of the window?

[AlexeyAB]

You can tweak this params: https://github.com/AlexeyAB/darknet/blob/89d0c46bb32a98416fb55a9c83c291dd0831c6cc/src/yolo_v2_class.hpp#L166-L168

• Window size should be equal to the average size of the object, so you can increase it if your objects larger than 21x21 pixels on the source video - it should be odd value: sync_PyrLKOpticalFlow_gpu->setWinSize(cv::Size(21, 21));
• Max level should be increased if you have long motions: sync_PyrLKOpticalFlow_gpu->setMaxLevel(3);
• Number of iterations should be increased for robust tracking: sync_PyrLKOpticalFlow_gpu->setNumIters(2000);

But the increase in these parameters will be slightly slow down the optical flow.

[xhuvom]

I am using GTX1080 GPU showing Detection ~55 FPS and Capture ~70 FPS on videofile. To test the tracking for object count, I have increased the parameters like this:
sync_PyrLKOpticalFlow_gpu = cv::cuda::SparsePyrLKOpticalFlow::create(); sync_PyrLKOpticalFlow_gpu->setWinSize(cv::Size(21, 21)); // 15, 21, 31 sync_PyrLKOpticalFlow_gpu->setMaxLevel(10); // +- 3 pt sync_PyrLKOpticalFlow_gpu->setNumIters(10000); // def: 30

However, its still not reliable enough. Probably its the fundamental issue with the YOLO algorithm itself because if I dump the detected frames, it shows that not all objects are detected on each single frame.

[AlexeyAB]

Current implementation of tracking only used for track objects only until next detection. So if on one of detections the object isn't found then it will not tracked longer. Later maybe I will add more longer tracking.

It is required only if you Вetection speed much slower than Capture speed.

[xhuvom]

Thanks for your helpful suggestions @AlexeyAB !!. Maybe Kalman filter could be added for more robust tracking if the object is missing at some frames but reappears again.

[AlexeyAB]

Yes, Kalman filter can be added, Kalman filter takes into account the physical inertia of the movement of objects. But I use this tracking for my task, where vido-camera moves very chaotically, so Kalman filter can't be used.

May be I will take bounded boxes from Tracking and from Detection, then I will concatenate them and do NMS (Non-maximum-supression), instead of replacing Tracking-box by Detection-box.
Also will try to filter tracking points by error and status: https://github.com/AlexeyAB/darknet/blob/89d0c46bb32a98416fb55a9c83c291dd0831c6cc/src/yolo_v2_class.hpp#L286

[yangulei]

@AlexeyAB Thanks for your excellent repo and your continuing update.
I encountered related errors while compiling "yolo_console_dll.sln". I’m using pre-build OpenCV3.4.0, CUDA8.0 and MSVS 2015.
I set the project to Release x64, and configure the OpenCV include and lib path. The compiling is stopped with error: Can't open "opencv_cudaoptflow340.lib".
Then I comment the "progma comment ..." lines below "opencv_world", no errors anymore and the .exe works fine. But I noticed that the output shows car ids, does this mean the tracking process is already working? I have not uncomment the line with “//#define TRACK_OPTFLOW” so far.
After uncommenting the line with “//#define TRACK_OPTFLOW”, the compiling is stopped with error said that file “opencv2/cudaoptflow.hpp” can’t be found. I search the OpenCV folder and there is indeed no "cudaoptflow.hpp". Should I compile OpenCV with MSVS?

[AlexeyAB]

@yangulei Hi,

• With commented //#define TRACK_OPTFLOW - there isn't optical flow tracking. There is only tracking_id() function, that shows track ID of objects.

• With un-commented #define TRACK_OPTFLOW - there is optical flow tracking, that helps to track objects in the realtime between detection. This is require if your GPU slow to detect objects for each frame in the realtime. To use #define TRACK_OPTFLOW you should compile your OpenCV 3.x with CUDA support, for this:

  • you should download Sources of the latest OpenCV 3.x: https://opencv.org/releases.html and put it to the directory C:/opencv_source/opencv (so cmake file CMakeLists.txt should be in the directory `C:/opencv_source/opencv/CMakeLists.txt)
  • you should install latest CMake: https://cmake.org/download/
  • run CMake, and enter where is source C:/opencv_source/opencv and where is binary C:/opencv_source/opencv/bin
  • then press Generate
  • then set to compile with CUDA and cudaoptflow, as in screenshots bellow
  • also disable options: BUILD_PERF_TESTS and BUILD_TESTS
  • then press Generate
  • then run MSVS2015, open C:\opencv_source\opencv\bin\OpenCV.sln and build it with x64 & Release

[yangulei]

@AlexeyAB Thanks for your timing and clear replay. I'm compiling the OpenCV, which is really a time consuming process.

update: arise C2610 errors, building with BUILD_PERF_TESTS and BUILD_TESTS disabled according to https://github.com/opencv/opencv/issues/6716

[AlexeyAB]

@yangulei Thanks, I added BUILD_PERF_TESTS and BUILD_TESTS to my answer.

[bit-scientist]

@AlexeyAB Cmake window, after pressing Generatebutton, I can't seem to find opencv_cudaoptflow option to tick

Should I put some cuda files inside my _opencv340/opencv/sources_?

[AlexeyAB]

@bit-scientist Set checkboxes grouped and advnaced, then enter to the CUDA-group

[bit-scientist]

Thank you, @AlexeyAB.
Still couldn't find. Because there's no CUDA group to enter to.
Maybe I should copy some cuda related files into _C:/opencv340/opencv/sources_?