Opencv_contrib: Cannot find NvOf library.

[Eyshika]

Using Samples of cudaoptflow, I am trying to run Nvidia Optical Flow sdk.

*code : *

 while(1)
    {
            Mat frame;
            cap >> frame; //read every frame
            if(frame.empty())
                break;
            Mat reference_image = Mat::zeros(frame.size(), frame.type());

            Ptr<NvidiaOpticalFlow_1_0> nvof = NvidiaOpticalFlow_1_0::create(
            frame.size().width, frame.size().height, perfPreset,
             true, false, false, gpuId);

            Mat flowx, flowy, flowxy, upsampledFlowXY, image;

            nvof->calc(frame, reference_image, flowxy);
            nvof->upSampler(flowxy, frame.size().width, frame.size().height,
                                        nvof->getGridSize(), upsampledFlowXY);

            Mat planes[] = { flowx, flowy };
        split(upsampledFlowXY, planes);
        flowx = planes[0]; flowy = planes[1];

        drawOpticalFlow(flowx, flowy, image, 10);
            outputv.write(image);
            char c = cv::waitKey(25);
            if (c ==27)
                    break;
            nvof->collectGarbage();
    }

Error on running:

OpenCV(4.1.2) /docker/opencv_contrib-4.1.2/modules/cudaoptflow/src/nvidiaOpticalFlow.cpp:172: error: (-6:Unknown error code -6) Cannot find NvOF library. in function 'LoadNvidiaModules'

whereas

define OF_MODULENAME "libnvidia-opticalflow.so.1"

found at

./lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1
./lib/i386-linux-gnu/libnvidia-opticalflow.so.1

System :

OPENCV - 4.1.2

Installed it inside docker container with cuda 10.1 : https://hub.docker.com/r/nvidia/cuda/
Image : nvidia/cuda
cuda_version : 10.1
OS: ubuntu18.04

@alalek

[alalek]

• Investigate: LD_DEBUG=files <app>
• LD_LIBRARY_PATH may help
• symlink (ln -s ...) may help
• ldconfig ?

---

Usage questions should go to Users OpenCV Q/A forum: http://answers.opencv.org

We don't help with remote investigation of problems which we are not able to reproduce on our side (because it is very time consuming and usually configuration problem without any following fix into OpenCV), but we can take a look on the clear reason of the problem and suggest some fix/workaround.

[Eyshika]

• Investigate: LD_DEBUG=files <app>
• LD_LIBRARY_PATH may help
• symlink (ln -s ...) may help
• ldconfig ?

Usage questions should go to Users OpenCV Q/A forum: http://answers.opencv.org

Even after ldconfig and creating opencv.conf file under /etc/ .. it throws same error

OpenCV(4.1.2) /docker/opencv_contrib-4.1.2/modules/cudaoptflow/src/nvidiaOpticalFlow.cpp:172: error: (-6:Unknown error code -6) Cannot find NvOF library. in function 'LoadNvidiaModules'

Is this something related to docker configuration , because when I do

find / -name "libnvidia-opticalflow.so.1"

inside docker container it doesnt give any result

[alalek]

libnvidia-opticalflow.so.1 is optional library (and probably can be installed through extra packages).
It is better to ask how to use this library in docker containers at NVIDIA dev forum: https://devtalk.nvidia.com/

We really can't fix misconfigured/misused docker containers from OpenCV library.

[Eyshika]

okay I understand . Thank you for prompt responses..
My last questions is just to know the exact reason behind this error am getting .. I mentioned about libnvidia-opticalflow.so.1, only because I found it mentioned inside /docker/opencv_contrib-4.1.2/modules/cudaoptflow/src/nvidiaOpticalFlow.cpp , where it is throwing an error and I already did symplink through ldconfig and opencv.conf file.

Again, the error is :

OpenCV(4.1.2) /docker/opencv_contrib-4.1.2/modules/cudaoptflow/src/nvidiaOpticalFlow.cpp:172: error: (-6:Unknown error code -6) Cannot find NvOF library. in function 'LoadNvidiaModules'

[vchiluka5]

Hi

NVIDIA optical flow library is packaged with NVIDIA graphics driver and it gets installed only if your GPU arch is Turing or above. Please check your GPU configuration and if its turing then pick the latest NVIDIA graphics driver from https://www.nvidia.com/en-us/drivers/unix/
For more information on NVIDIA Optical Flow SDK, Please refer https://developer.nvidia.com/opticalflow-sdk

[Eyshika]

Hi @vchiluka5
Thanx for your answer, I did check the documentaion and I am using turing Card with latest Nvidia driver, still am getting this error.
Here is my system description:

| NVIDIA-SMI 435.21       Driver Version: 435.21       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:0A:00.0 Off |                  N/A |
| 41%   33C    P8    21W / 260W |      1MiB / 11019MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce RTX 208...  Off  | 00000000:41:00.0 Off |                  N/A |
| 41%   36C    P8    20W / 260W |     26MiB / 11016MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

[vchiluka5]

@Eyshika System configuration looks fine . Sorry i missed that you are using docker container. Docker support is not yet implemented wrt NVOF library. Its still under progress. Can you please try it outside docker in a standalone machine?

[vchiluka5]

Hi
What i found is:
nvidia-docker (or nvidia-docker2) is just a script now. Any image involving CUDA will depend on the base CUDA images (like the ones at https://hub.docker.com/r/nvidia/cuda/), which require the libnvidia-container and nvidia-container runtime packages to be installed and libnvidia-container package should be above v1.0.2 for NVIDIA Optical flow libraries to be visible to docker. These packages must be present on host machine which makes NVOF library visible to docker.

[Eyshika]

Hi @vchiluka5 :

As you suggested, to use standalone system instead of dokcer.. am trying to use that and read a video file from command using

i=

now it throws error for gpuMat

OpenCV(4.1.2) /home/opencv-4.1.2/modules/core/src/cuda/gpu_mat.cu:224: error: (-217:Gpu API call) invalid pitch argument in function 'upload'

[Eyshika]

nvidia-container runtime packages

regarding nvidia container packages, I did

sudo apt-get install nvidia-container-runtime

and system already has these packages installed

[vchiluka5]

OpenCV(4.1.2) /home/opencv-4.1.2/modules/core/src/cuda/gpu_mat.cu:224: error: (-217:Gpu API call) invalid pitch argument in function 'upload'

Issue seems to be not related with NVOF. Can you please share your sample app? Let me try it out.

[Eyshika]

@vchiluka5 : I figured out the issue for standalone system and its running now.. but still trying to figure out the docker way (if the file has already been integrated)

[vchiluka5]

@Eyshika Good to hear that you made it working in non docker environment. I got a repro with docker and even i am getting same error even when nvidia-runtime/ nvidia-container version is > 1.0.2 . I have escalated it to NVIDIA docker team and will surely resolve it within no time. Will let you know once it gets fixed.
Thanks :-)

[vchiluka5]

@Eyshika
You need to add "-e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility" while running docker
This is used by our container library component (libnvidia-container) to expose the video libraries (encode, decode, optical flow) inside the container.
I think the only place where this additional option to pass in environment variables is documented is on the GitHub pages. We will add this to OpenCV also.

so your command should look like
docker run --ipc=host -p 6007:6007 --runtime=nvidia --security-opt seccomp=unconfined -it -e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility --rm nvidia/cuda

Tested it on my machine and libnvidia-opticalflow is now visible to docker.
You can go through this link for in depth details: https://github.com/NVIDIA/nvidia-container-runtime/blob/2c07b861b81afd4af683a0c1e71a250f1e2e3344/README.md .