Pybind11: numpy opencv convertor

[peendebak]

Is there any option to wrap functions that accept or return an OpenCV cv::Mat object?

For example suppose I have a C++ function like

double myFunction(const cv::Mat &m, double parameter);

I would like to be able to call from Python

m = numpy.random.rand( (20, 30) )
result = myclass.myFunction(m, 2.0)

[wjakob]

No, there isn't. Please refer to the docs on binding matrix-like types.

[peendebak]

@wjakob In the docs I can find how to use the py::buffer interface to wrap my function, this is already a good first step.

I cannot find how to create transparent conversion for OpenCV matrices, such as described in https://pybind11.readthedocs.io/en/latest/advanced.html#transparent-conversion-of-dense-and-sparse-eigen-data-types for Eigen matrices.

What would it take to create such transparent conversion? Would it be enough to adapt https://github.com/pybind/pybind11/blob/master/include/pybind11/eigen.h to OpenCV style matrices?

[wjakob]

Sadly I don't have the resources to support pybind11 integration with other libraries.If you really want this, you could study the eigen integration and try to replicate it, but I can't provide any support here (i.e. you're on your own.)

[peendebak]

Ok, thanks for the reply.

In #include <opencv2/core/eigen.hpp> there are converters for Eigen to OpenCV. For now I will use the transparent conversion to Eigen and wrap by functions manually.

[jampekka]

As this is the first hit in google for pybind11 opencv, here's a quick (and probably buggy) opencv <-> numpy converter. Supports only float arrays now, but adding more types is trivial.

#include <pybind11/numpy.h>
#include <opencv2/core/core.hpp>

namespace pybind11 { namespace detail {

template <> struct type_caster<cv::Mat> {

    bool load(handle src, bool) {
        array b(src, true);
        if(!b.check()) return false;
        auto info = b.request();

        decltype(CV_32F) dtype;
        if(info.format == format_descriptor<float>::value) dtype = CV_32F;
        else if (info.format == format_descriptor<double>::value) dtype = CV_64F;
        else return false;

        auto ndims = info.ndim;
        auto shape = std::vector<int>(info.shape.begin(), info.shape.end());
        auto& strides = info.strides;
        value = cv::Mat(ndims,
                &shape[0],
                dtype,
                info.ptr,
                &strides[0]);
        return true;

    }

    static handle cast(const cv::Mat &m, return_value_policy, handle defval) {
        auto format = format_descriptor<float>::value;
        auto type = m.type();
        switch(type) {
            case CV_32F: format = format_descriptor<float>::value; break;
            case CV_64F: format = format_descriptor<double>::value; break;
            default: return defval;
        }

        std::vector<size_t> IHateBjarneStroustrup;
        std::copy(m.size.p, m.size.p + m.dims, std::back_inserter(IHateBjarneStroustrup));
        auto strides = std::vector<size_t>(m.step.p, m.step.p + m.dims);
        strides.push_back(1);
        return array(buffer_info(
            m.data,
            m.elemSize1(),
            format,
            m.dims,
            IHateBjarneStroustrup,
            strides
            )).release();
    }

    PYBIND11_TYPE_CASTER(cv::Mat, _("array"));
};
}}

[edmBernard]

I link here a my working solution :
https://github.com/pybind/pybind11/issues/538#issuecomment-263884464
It's less buggy than @jampekka's answer but it can really be improve

[virtuald]

@jampekka and @edmBernard see https://github.com/pybind/pybind11/issues/538#issuecomment-273981569 . I copy the OpenCV numpy allocator code, it should be fairly optimal.

[Coderx7]

For sending a cv::Mat from C++ to Python see this : https://stackoverflow.com/questions/60949451/how-to-send-a-cvmat-to-python-over-shared-memory

For sending a cv image from Python to C++ see this : https://stackoverflow.com/questions/60917800/how-to-get-the-opencv-image-from-python-and-use-it-in-c-in-pybind11

[edmBernard]

btw, The solution proposed by @virtuald that I use for 3 years now, was implemented here https://github.com/edmBernard/pybind11_opencv_numpy