Mask_rcnn: save visualized image

[NikolayStarikov]

Hi guys,
Could you describe how to save image after visualize.display_instances () plot it in jupyter notebook.
I'd like to save it automaticaly.

[amlarraz]

Hi @NikolayStarikov ,

I've added the following lines in visualize.py and it works:(lineas 146, 147)

plt.savefig('your_desired_path_to_image.jpg')
plt.close()

[larry10hhobh]

Hi @amlarraz ,
I use
plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight')
but it still hav not exact size with some blank.
how do you solve it?Thx

[baolinliu]

I had the same problem too. I used: plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape') , the key is negative padding, which worked for me.

[cvtower]

I had the same problem too. I used: plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape') , the key is negative padding, which worked for me.

Nice solution, works for me!

[amlarraz]

Hi! All problem comes with the utilization of matplotlib to create the visualization. I'm working on a function that will save the original image with painted bboxes.

[amlarraz]

I've finished the funtion to show the results and save them in the original size of the image if you want:

def display_results(image, boxes, masks, class_ids, class_names, scores=None,
                        show_mask=True, show_bbox=True, display_img=True,
                        save_img=True, save_dir=None, img_name=None):
        """
        boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
        masks: [height, width, num_instances]
        class_ids: [num_instances]
        class_names: list of class names of the dataset (Without Background)
        scores: (optional) confidence scores for each box
        show_mask, show_bbox: To show masks and bounding boxes or not
        display_img: To display the image in popup
        save_img: To save the predict image
        save_dir: If save_img is True, the directory where you want to save the predict image
        img_name: If save_img is True, the name of the predict image

        """
        n_instances = boxes.shape[0]
        colors = color_map()
        for k in range(n_instances):
            color = colors[class_ids[k]].astype(np.int)
            if show_bbox:
                box = boxes[k]
                cls = class_names[class_ids[k]-1]  # Skip the Background
                score = scores[k]
                cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1)
                font = cv2.FONT_HERSHEY_SIMPLEX
                cv2.putText(image, '{}: {:.3f}'.format(cls, score), (box[1], box[0]),
                            font, 0.4, (0, 255, 255), 1, cv2.LINE_AA)

            if show_mask:
                mask = masks[:, :, k]
                color_mask = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.int)
                color_mask[mask] = color
                image = cv2.addWeighted(color_mask, 0.5, image.astype(np.int), 1, 0)

        if display_img:
            plt.imshow(image)
            plt.xticks([]), plt.yticks([])  # to hide tick values on X and Y axis
            plt.show()
        if save_img:
            cv2.imwrite(os.path.join(save_dir, img_name), image)

        return None

To choose the color, I've used the function that creates the colormap for PascalVOC, if you want to use the function above you will need the function below:

def color_map(N=256, normalized=False):
        def bitget(byteval, idx):
            return ((byteval & (1 << idx)) != 0)

        dtype = 'float32' if normalized else 'uint8'
        cmap = np.zeros((N, 3), dtype=dtype)
        for i in range(N):
            r = g = b = 0
            c = i
            for j in range(8):
                r = r | (bitget(c, 0) << 7 - j)
                g = g | (bitget(c, 1) << 7 - j)
                b = b | (bitget(c, 2) << 7 - j)
                c = c >> 3

            cmap[i] = np.array([r, g, b])

        cmap = cmap / 255 if normalized else cmap
        return cmap

An example of use:

display_results(image, r['rois'], r['masks'], r['class_ids'], classes, r['scores'], save_dir="/home/user/inferences", img_name="inferenced.png")

[frankdeepl]

Hi @amlarraz ,
I use
plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight')
but it still hav not exact size with some blank.
how do you solve it?Thx

Hi @larry10hhobh,
I tried to use your method to save image, how I just get the blank image. Do you know what is the problem here?

[L7c8ana]

if using
import matplotlib
# Agg backend runs without a display
matplotlib.use('Agg')
import matplotlib.pyplot as plt
use plt.savefig('your_desired_path_to_image.pdf')

[AhmadShaik]

I had the same problem too. I used: plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape') , the key is negative padding, which worked for me.

Can you tell me where you have added that line. Is it in visualize.py script or in jupyter script

[ZhiyuanChen]

I've finished the funtion to show the results and save them in the original size of the image if you want:

def display_results(image, boxes, masks, class_ids, class_names, scores=None,
                        show_mask=True, show_bbox=True, display_img=True,
                        save_img=True, save_dir=None, img_name=None):
        """
        boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
        masks: [height, width, num_instances]
        class_ids: [num_instances]
        class_names: list of class names of the dataset (Without Background)
        scores: (optional) confidence scores for each box
        show_mask, show_bbox: To show masks and bounding boxes or not
        display_img: To display the image in popup
        save_img: To save the predict image
        save_dir: If save_img is True, the directory where you want to save the predict image
        img_name: If save_img is True, the name of the predict image

        """
        n_instances = boxes.shape[0]
        colors = color_map()
        for k in range(n_instances):
            color = colors[class_ids[k]].astype(np.int)
            if show_bbox:
                box = boxes[k]
                cls = class_names[class_ids[k]-1]  # Skip the Background
                score = scores[k]
                cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1)
                font = cv2.FONT_HERSHEY_SIMPLEX
                cv2.putText(image, '{}: {:.3f}'.format(cls, score), (box[1], box[0]),
                            font, 0.4, (0, 255, 255), 1, cv2.LINE_AA)

            if show_mask:
                mask = masks[:, :, k]
                color_mask = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.int)
                color_mask[mask] = color
                image = cv2.addWeighted(color_mask, 0.5, image.astype(np.int), 1, 0)

        if display_img:
            plt.imshow(image)
            plt.xticks([]), plt.yticks([])  # to hide tick values on X and Y axis
            plt.show()
        if save_img:
            cv2.imwrite(os.path.join(save_dir, img_name), image)

        return None

To choose the color, I've used the function that creates the colormap for PascalVOC, if you want to use the function above you will need the function below:

def color_map(N=256, normalized=False):
        def bitget(byteval, idx):
            return ((byteval & (1 << idx)) != 0)

        dtype = 'float32' if normalized else 'uint8'
        cmap = np.zeros((N, 3), dtype=dtype)
        for i in range(N):
            r = g = b = 0
            c = i
            for j in range(8):
                r = r | (bitget(c, 0) << 7 - j)
                g = g | (bitget(c, 1) << 7 - j)
                b = b | (bitget(c, 2) << 7 - j)
                c = c >> 3

            cmap[i] = np.array([r, g, b])

        cmap = cmap / 255 if normalized else cmap
        return cmap

An example of use:

display_results(image, r['rois'], r['masks'], r['class_ids'], classes, r['scores'], save_dir="/home/user/inferences", img_name="inferenced.png")

Thank you for your reply, but it seems your code is not working on my laptop.
It raise TypeError: only size-1 arrays can be converted to Python scalars at cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1). Which is probably because the type of color is ndarray. Therefore, cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color.tolist(), 1) can be a better approach.
Besides, I don't think plt.imshow(image) will not raise an exception, but anyway, thank you very much.

[mehdiselbi]

I've finished the funtion to show the results and save them in the original size of the image if you want:

def display_results(image, boxes, masks, class_ids, class_names, scores=None,
                        show_mask=True, show_bbox=True, display_img=True,
                        save_img=True, save_dir=None, img_name=None):
        """
        boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
        masks: [height, width, num_instances]
        class_ids: [num_instances]
        class_names: list of class names of the dataset (Without Background)
        scores: (optional) confidence scores for each box
        show_mask, show_bbox: To show masks and bounding boxes or not
        display_img: To display the image in popup
        save_img: To save the predict image
        save_dir: If save_img is True, the directory where you want to save the predict image
        img_name: If save_img is True, the name of the predict image

        """
        n_instances = boxes.shape[0]
        colors = color_map()
        for k in range(n_instances):
            color = colors[class_ids[k]].astype(np.int)
            if show_bbox:
                box = boxes[k]
                cls = class_names[class_ids[k]-1]  # Skip the Background
                score = scores[k]
                cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1)
                font = cv2.FONT_HERSHEY_SIMPLEX
                cv2.putText(image, '{}: {:.3f}'.format(cls, score), (box[1], box[0]),
                            font, 0.4, (0, 255, 255), 1, cv2.LINE_AA)

            if show_mask:
                mask = masks[:, :, k]
                color_mask = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.int)
                color_mask[mask] = color
                image = cv2.addWeighted(color_mask, 0.5, image.astype(np.int), 1, 0)

        if display_img:
            plt.imshow(image)
            plt.xticks([]), plt.yticks([])  # to hide tick values on X and Y axis
            plt.show()
        if save_img:
            cv2.imwrite(os.path.join(save_dir, img_name), image)

        return None

To choose the color, I've used the function that creates the colormap for PascalVOC, if you want to use the function above you will need the function below:

def color_map(N=256, normalized=False):
        def bitget(byteval, idx):
            return ((byteval & (1 << idx)) != 0)

        dtype = 'float32' if normalized else 'uint8'
        cmap = np.zeros((N, 3), dtype=dtype)
        for i in range(N):
            r = g = b = 0
            c = i
            for j in range(8):
                r = r | (bitget(c, 0) << 7 - j)
                g = g | (bitget(c, 1) << 7 - j)
                b = b | (bitget(c, 2) << 7 - j)
                c = c >> 3

            cmap[i] = np.array([r, g, b])

        cmap = cmap / 255 if normalized else cmap
        return cmap

An example of use:
display_results(image, r['rois'], r['masks'], r['class_ids'], classes, r['scores'], save_dir="/home/user/inferences", img_name="inferenced.png")

Thank you for your reply, but it seems your code is not working on my laptop.
It raise TypeError: only size-1 arrays can be converted to Python scalars at cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1). Which is probably because the type of color is ndarray. Therefore, cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color.tolist(), 1) can be a better approach.
Besides, I don't think plt.imshow(image) will not raise an exception, but anyway, thank you very much.

same thing here
i think its because of cv2 behaviour
@amlarraz can you help

[ZhiyuanChen]

I've finished the funtion to show the results and save them in the original size of the image if you want:

def display_results(image, boxes, masks, class_ids, class_names, scores=None,
                        show_mask=True, show_bbox=True, display_img=True,
                        save_img=True, save_dir=None, img_name=None):
        """
        boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
        masks: [height, width, num_instances]
        class_ids: [num_instances]
        class_names: list of class names of the dataset (Without Background)
        scores: (optional) confidence scores for each box
        show_mask, show_bbox: To show masks and bounding boxes or not
        display_img: To display the image in popup
        save_img: To save the predict image
        save_dir: If save_img is True, the directory where you want to save the predict image
        img_name: If save_img is True, the name of the predict image

        """
        n_instances = boxes.shape[0]
        colors = color_map()
        for k in range(n_instances):
            color = colors[class_ids[k]].astype(np.int)
            if show_bbox:
                box = boxes[k]
                cls = class_names[class_ids[k]-1]  # Skip the Background
                score = scores[k]
                cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1)
                font = cv2.FONT_HERSHEY_SIMPLEX
                cv2.putText(image, '{}: {:.3f}'.format(cls, score), (box[1], box[0]),
                            font, 0.4, (0, 255, 255), 1, cv2.LINE_AA)

            if show_mask:
                mask = masks[:, :, k]
                color_mask = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.int)
                color_mask[mask] = color
                image = cv2.addWeighted(color_mask, 0.5, image.astype(np.int), 1, 0)

        if display_img:
            plt.imshow(image)
            plt.xticks([]), plt.yticks([])  # to hide tick values on X and Y axis
            plt.show()
        if save_img:
            cv2.imwrite(os.path.join(save_dir, img_name), image)

        return None

To choose the color, I've used the function that creates the colormap for PascalVOC, if you want to use the function above you will need the function below:

def color_map(N=256, normalized=False):
        def bitget(byteval, idx):
            return ((byteval & (1 << idx)) != 0)

        dtype = 'float32' if normalized else 'uint8'
        cmap = np.zeros((N, 3), dtype=dtype)
        for i in range(N):
            r = g = b = 0
            c = i
            for j in range(8):
                r = r | (bitget(c, 0) << 7 - j)
                g = g | (bitget(c, 1) << 7 - j)
                b = b | (bitget(c, 2) << 7 - j)
                c = c >> 3

            cmap[i] = np.array([r, g, b])

        cmap = cmap / 255 if normalized else cmap
        return cmap

An example of use:
display_results(image, r['rois'], r['masks'], r['class_ids'], classes, r['scores'], save_dir="/home/user/inferences", img_name="inferenced.png")

Thank you for your reply, but it seems your code is not working on my laptop.
It raise TypeError: only size-1 arrays can be converted to Python scalars at cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color, 1). Which is probably because the type of color is ndarray. Therefore, cv2.rectangle(image, (box[1], box[0]), (box[3], box[2]), color.tolist(), 1) can be a better approach.
Besides, I don't think plt.imshow(image) will not raise an exception, but anyway, thank you very much.

same thing here
i think its because of cv2 behaviour
@amlarraz can you help

Nah, imshow is not a plt function. I would recommend to just delete all those things. Here is an example:
def draw_result(image, result, show_mask=True, show_bbox=True): colors = color_map() for i in range(result['rois'].shape[0]): color = colors[result['class_ids'][i]].astype(np.int).tolist() if show_bbox: coordinate = result['rois'][i] cls = class_names[result['class_ids'][i] - 1] score = result['scores'][i] cv2.rectangle(image, (coordinate[1], coordinate[0]), (coordinate[3], coordinate[2]), color, 1) font = cv2.FONT_HERSHEY_SIMPLEX cv2.putText(image, '{}: {:.3f}'.format(cls, score), (coordinate[1], coordinate[0]), font, 0.4, (0, 255, 255), 1, cv2.LINE_AA) if show_mask: mask = result['masks'][:, :, i] color_mask = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.int) color_mask[mask] = color image = cv2.addWeighted(color_mask, 0.5, image.astype(np.int), 1, 0) return image
This function would return the result in ndarray wehre you can use cv.imwrite or cv.imshow or whatever.

[siddheshgunjal]

I had the same problem too. I used: plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape') , the key is negative padding, which worked for me.

where to add this line in visualize.py?

[kientruccapa]

This work for me #1619

[hkmurali]

bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape')

U saved my life baolinliu :)

[fvangef]

so I tried to add baolinliu 's line in the visualize.py, but nothing happens...

plt.savefig('/path/to/image/mage1.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape')

is it correct that I add it after:
ax.imshow(masked_image.astype(np.uint8))

and before:
if auto_show:
plt.show()

In def display_instances?

Would reallt appreciate the help!

[Nikk-27]

@fvangef did you get to know where to write that line (save_image)