Opencv_contrib: Documentation for CUDA Python modules in 4.0.0?

[nhubbard]

System information

• OpenCV => 4.0.1
• Operating System / Platform => Linux 64-Bit, with CUDA and MKL
• Compiler => gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0

Detailed description

I found this article in search of CUDA and Python support for OpenCV. https://jamesbowley.co.uk/build-opencv-4-0-0-with-cuda-10-0-and-intel-mkl-tbb-in-windows/
Unfortunately, I have run into some implementation problems, namely that the entire cv2.cuda namespace appears to be undocumented or broken. Converting from Mat to GpuMat works, but converting back to Mat is seemingly impossible.

Steps to reproduce

# First, build OpenCV 4.0.1 with CUDA support (MKL not necessary)
import cv2
status, result = cv2.VideoCapture(0).read()
# Now, convert input to GpuMat to use with `cv2.cuda` functions.
result_cuda = cv2.cuda_GpuMat(result)
# But now this error occurs.
cv2.cuda.resize(result_cuda, (300, 300))
# TypeError: Expected cv::UMat for argument 'src'
# So we convert to UMat.
result_umat = cv2.UMat(result)
cv2.cuda.resize(result_umat, (300, 300))
# Now we get a conflicting error:
# error: OpenCV(4.0.1) /home/nvidia/opencv/modules/core/src/matrix_wrap.cpp:359: error: (-213:The function/feature is not implemented) getGpuMat is available only for cuda::GpuMat and cuda;:HostMem in function `getGpuMat`
# Okay, so we now explicitly wrap it...?
cv2.cuda.resize(cv2.cuda_GpuMat(result_umat), (300, 300))
# Result: <cuda_GpuMat 0x7f3567b57df0>
# Now try to show the resized image.
cv2.imshow("Out", cv2.cuda.resize(cv2.cuda_GpuMat(result_umat), (300, 300)))
# TypeError: Expected cv::UMat for argument 'mat'
# Okay, so try to convert the result to UMat?
cv2.imshow("Out", cv2.UMat(cv2.cuda.resize(cv2.cuda_GpuMat(result_umat), (300, 300))))
# TypeError: Required argument 'ranges' (pos 2) not found
# This really messed with me and I can't find the type for the argument, resulting in a blocker.
# So I try a test, and sure enough, even supplying a ranges argument fails:
cuda = cv2.cuda_GpuMat(result)
cv2.UMat(cuda)
# TypeError: Required argument 'ranges' (pos 2) not found
cv2.UMat(cuda, ranges=0)
# TypeError: Expected cv::UMat for argument 'm'
# Where did this argument 'm' come from?
cv2.UMat(cuda, cv2.UMat())
# TypeError: Expected cv::UMat for argument 'm'
# I give up.

[alalek]

Reference: tests with Python CUDA code.

[nhubbard]

That link is very helpful for most functionality, but is there a way to convert a GpuMat to a UMat or regular Mat/Numpy array? I need this because I need to run TensorFlow inference on the processed image.

[cudawarped]

The CUDA python bindings are very new, (08/2018 if i remember correctly) and not complete but they are a good start. Hopefully the more people that use them the better they will get.

From memory to get a numpy array you should be able to call download, e.g. result_cuda.download(). Also whenever i saw umat errors it had nothing to do with umat and was because the arguments i was passing were incorrect. For example you may need to pass the destination gpu mat as the second argument to the resize function in your example. You can check the header files in build/modules/python/ for the function signatures (for more info see https://docs.opencv.org/3.2.0/da/d49/tutorial_py_bindings_basics.html. I think each function has a default umat signature if all others are not matched. Unfortunately i am away from my laptop so i cant check but hopefully the above will help.