Opencv_contrib: selectROI pointer almost invisible on certain colours

[JeongJuhyeon]

System information (version)

• OpenCV => 3.4.2.17
• Operating System / Platform => Win 10 64-Bit
• Compiler => Python 3 (opencv-python)

Detailed description

When using selectROI you get a crosshair pointer that seemingly changes colour depending on the background behind it. Problem is that with certain background colours it's almost impossible to see.
The pointer is inside the circle, you might have to zoom in to even see it.

Steps to reproduce

Use the selectROI function. Mouse over a greyish background.

[alalek]

Mentioned code is here.
Feel free to propose changes into 3.4 branch.

[JeongJuhyeon]

That code seems to apply to the "crosshair in the middle of bounding box", not to the mouse pointer? Maybe the code setting the mouse pointer appearance is in cv2.imshow?

[parthpankajtiwary]

Hi @JeongJuhyeon!

I am unable to find the code for the cross-hair pointer adapting color depending on the background. Can you please include a reference for the code snippet which is responsible for this behavior besides the one which @alalek has referenced above? :)

[berak]

let me clarify:
the problem is neither with cv2(python code), nor with selectRoi()(c++ code), but with the mouse cursor.

by default, the "precision selection" cursor is used on windows, and this tries to invert the colors beneath it, which obviously works bad, for a uniform gray surface.

to change that, you'd have to work on highgui/src/window_w32.cpp, and add an option to change the mouse cursor there.

but i think, this is clumsy. maybe we can just add documentation, how to change the cursor from the user's system panel (manually) instead.

[hkhojasteh]

@berak I'm interested in solving this problem. How can I start working on this version?

[berak]

hi, @hkhojasteh :

let me be honest: i don't think, it's a good idea to change the library code here. (as in: window_w32.cpp)
(it's not really a relevant problem, imho, and can be overcome by simple means, like not using a uniform gray bg, or changing the cursor from the system panel)

(nice, though, that you're trying to help, !)

[sturkmen72]

@hkhojasteh you can try to add an horizontal and vertical (w=img.cols h=img.rows ) line which its center is the mouse pointer. i think it will be helpful.

[hkhojasteh]

@sturkmen72 Can you suggest me the related file to start fixing this issue?

[sturkmen72]

as mentioned before the code is here

also see my crude attempt to improve ROISelector class

#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include <iostream>

using namespace cv;

class ROISelector
{
  public:
    Rect select(const String &caption, InputArray _img, bool _showCrosshair = true, bool fromCenter = true)
    {
        // show notice to user
        printf("Select a ROI and then press SPACE or ENTER button!\n");
        printf("Cancel the selection process by pressing c button!\n");

        key = 0;
        img = _img.getMat();
        imageSize = img.size();
        windowName = caption;
        showCrosshair = _showCrosshair;

        // set the drawing mode
        selectorParams.drawFromCenter = fromCenter;

        // show the image and give feedback to user
        imshow(windowName, img);

        // copy the data, rectangle should be drawn in the fresh image
        selectorParams.image = img.clone();

        // select the object
        setMouseCallback(windowName, mouseHandler, (void*)this);

        // end selection process on SPACE (32) ESC (27) or ENTER (13)
        while (!(key == 32 || key == 27 || key == 13))
        {
            // get keyboard event
            key = waitKey(0);

            if (key == 'c' || key == 'C')//cancel selection
            {
                selectorParams.box = Rect();
                break;
            }
        }

        //cleanup callback
        setMouseCallback(windowName, emptyMouseHandler, NULL);

        return selectorParams.box;
    }

    void select(const String &caption, InputArray _img, std::vector<Rect> &boundingBoxes,
                bool _showCrosshair = true, bool fromCenter = true)
    {
        printf("Finish the selection process by pressing ESC button!\n");
        boundingBoxes.clear();
        key = 0;

        // while key is not ESC (27)
        for (;;)
        {
            Rect temp = select(caption, _img, _showCrosshair, fromCenter);
            if (key == 27)
                break;
            if (temp.width > 0 && temp.height > 0)
                boundingBoxes.push_back(temp);
        }
    }

    struct handlerT
    {
        // basic parameters
        bool isDrawing;
        Rect2d box;
        Mat image;

        // parameters for drawing from the center
        bool drawFromCenter;
        Point2f center;

        // initializer list
        handlerT() : isDrawing(false), drawFromCenter(true){};
    } selectorParams;

  private:
    static void emptyMouseHandler(int, int, int, int, void*)
    {
    }

    static void mouseHandler(int event, int x, int y, int flags, void *param)
    {
        ROISelector *self = static_cast<ROISelector *>(param);
        self->opencv_mouse_callback(event, x, y, flags);
    }

    void opencv_mouse_callback(int event, int x, int y, int flags)
    {
        mPoint.x = x;
        mPoint.y = y;

        switch (event)
        {
        // update the selected bounding box
        case EVENT_MOUSEMOVE:
            if (selectorParams.isDrawing)
            {
                if (selectorParams.drawFromCenter)
                {
                    selectorParams.box.width = 2 * (x - selectorParams.center.x);
                    selectorParams.box.height = 2 * (y - selectorParams.center.y);
                    selectorParams.box.x = std::min(
                                std::max(selectorParams.center.x - selectorParams.box.width / 2.0, 0.), (double)imageSize.width);
                    selectorParams.box.y = std::min(
                                std::max(selectorParams.center.y - selectorParams.box.height / 2.0, 0.), (double)imageSize.height);
                }
                else
                {
                    selectorParams.box.width = std::max(
                                std::min(x - selectorParams.box.x, (double)imageSize.width - selectorParams.box.x), - selectorParams.box.x);
                    selectorParams.box.height = std::max(
                                std::min(y - selectorParams.box.y, (double)imageSize.height - selectorParams.box.y), - selectorParams.box.y);
                }
            }
            break;

        // start to select the bounding box
        case EVENT_LBUTTONDOWN:
            selectorParams.isDrawing = true;
            selectorParams.box = Rect2d(x, y, 0, 0);
            selectorParams.center = Point2f((float)x, (float)y);
            break;

        // cleaning up the selected bounding box
        case EVENT_LBUTTONUP:
            selectorParams.isDrawing = false;
            if (selectorParams.box.width < 0)
            {
                selectorParams.box.x += selectorParams.box.width;
                selectorParams.box.width *= -1;
            }
            if (selectorParams.box.height < 0)
            {
                selectorParams.box.y += selectorParams.box.height;
                selectorParams.box.height *= -1;
            }
            break;
        }

        if (mPoint.inside(Rect(0, 0, selectorParams.image.cols, selectorParams.image.rows)))
        {
            selectorParams.image = img.clone();

            if(flags & EVENT_FLAG_SHIFTKEY)
            {
                Rect rv(mPoint.x, 0, 1, selectorParams.image.rows);
                Rect rh(0, mPoint.y, selectorParams.image.cols, 1);

                Mat vline = selectorParams.image(rv);
                Mat hline = selectorParams.image(rh);

                vline = vline*3;
                hline = hline * 3;
            }

            // draw the selected object
            rectangle(selectorParams.image, selectorParams.box, Scalar(255, 0, 0), 2, 1);

            // draw cross air in the middle of bounding box
            if (showCrosshair)
            {
                // horizontal line
                line(selectorParams.image,
                     Point((int)selectorParams.box.x,
                           (int)(selectorParams.box.y + selectorParams.box.height / 2)),
                     Point((int)(selectorParams.box.x + selectorParams.box.width),
                           (int)(selectorParams.box.y + selectorParams.box.height / 2)),
                     Scalar(255, 0, 0), 2, 1);

                // vertical line
                line(selectorParams.image,
                     Point((int)(selectorParams.box.x + selectorParams.box.width / 2),
                           (int)selectorParams.box.y),
                     Point((int)(selectorParams.box.x + selectorParams.box.width / 2),
                           (int)(selectorParams.box.y + selectorParams.box.height)),
                     Scalar(255, 0, 0), 2, 1);
            }

            imshow(windowName, selectorParams.image);
        }
    }

    Mat img;
    String windowName;
    bool showCrosshair;
    int key;         // save the keypressed character
    Size imageSize;
    Point mPoint;
};

int main()
{
    Mat image = imread("../data/lena.jpg");
    ROISelector selector;
    selector.select("ROI selector", image, true, false);
    return 0;
}

[sturkmen72]

@hkhojasteh @berak how can we work collaboratively on improving the code?

[sturkmen72]

@hkhojasteh @berak @sovrasov i updated the code here. lines are visible when the Shift key is pressed. any remark ?

[berak]

@sturkmen72 -- perfect !

[alalek]

I believe we can draw lines without pressed Shift key (and hide them when Shift key is pressed)