Models: utils: detected object labels are sometimes not visible on the image

[frederictost]

Issue:
Sometimes the object labels added over an image are out of image bounding. For example, see function visualize_boxes_and_labels_on_image_array

Solution:
Draw the label at the bottom of the detected object bounding box. The file to be modified is:
visualization_utils.py

# draw.rectangle(
#     [(left, text_bottom - text_height - 2 * margin), (left + text_width,
#                                                       text_bottom)],
#     fill=color)

# draw.text(
#     (left + margin, text_bottom - text_height - margin),
#     display_str,
#     fill='black',
#     font=font)
# text_bottom -= text_height - 2 * margin

# Modified by Frederic TOST
# Compute a height ratio, if ratio < 10% put the label at the bottom of rectangle
if (top / im_height < 0.1):
    draw.rectangle(
        [(left, bottom + text_height + 2 * margin), (left + text_width,
                                                     bottom)],
        fill=color)
    draw.text(
        (left + margin, bottom),
        display_str,
        fill='black',
        font=font)
    text_bottom += text_height + 2 * margin
else:
    draw.rectangle(
        [(left, text_bottom - text_height - 2 * margin), (left + text_width,
                                                          text_bottom)],
        fill=color)
    draw.text(
        (left + margin, text_bottom - text_height - margin),
        display_str,
        fill='black',
        font=font)
    text_bottom -= text_height - 2 * margin

[angerson]

Thanks for the work you've done so far. Can you submit a pull request with your change?

[frederictost]

Hi Austin,

I'm working on a better solution (algo to find the better location for
labels), to make it simple I will change only the file
visualization_utils.py

I don't know how to submit a pull request to the community before
validation (may be I can find on GitHub).

Frederic

2017-11-11 2:58 GMT+01:00 Austin Anderson [e-mail]:

Thanks for the work you've done so far. Can you submit a pull request with
your change?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/models/issues/2771#issuecomment-343631608,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALrNiALlsvRFgE4cjyXe1dGALz8XzCZ9ks5s1P84gaJpZM4QaSQR
.

[frederictost]

I've published a pull request to fix this issue.

[tatatodd]

@frederictost Thanks for the work! Can you add a link to your PR in the comments here?

Also marking this as contributions welcome, since you're already working on this.

[cindysillence]

I just change the font size "font = ImageFont.truetype('arial.ttf', 32)" but the labels still as same as usual . how I can do it ?

[frederictost]

The Pull Request is #2805

[frederictost]

"font = ImageFont.truetype('arial.ttf', 32)"
Which line from which file did you changed ?

[cindysillence]

@frederictost i just change the number from 16 to 32 . The label should be more biger . but it still am small as usual .

[frederictost]

With my new code I used the option font_size = 42
vis_util.visualize_boxes_and_labels_on_image_array(
image_np,
np.squeeze(boxes),
np.squeeze(classes).astype(np.int32),
np.squeeze(scores),
category_index,
use_normalized_coordinates=True,
line_thickness=0,
font_size = 42)
It works on Windows 8. I did not try on Linux.

If you don't set the font size it is automatically computed.

[JunshengFu]

I am running this code in Ubuntu16.04.
At file visualization_utils.py, with following commands, I only get the default font size. It seems that there is no 'arial.ttf'.

  try:
    font = ImageFont.truetype('arial.ttf', 24)
  except IOError:
    font = ImageFont.load_default()

[frederictost]

Hello, you are right, under Ubuntu I've just had some issues and fixed the code, see the Pull Request #2805
Feel free to ask if any question.

[ozett]

for an (temporary) ubuntu solution to missing arial.ttf go here:
https://stackoverflow.com/questions/47242183/customizing-tf-object-detection-bounding-box-thickness-label-font-size

However you can workaround this problem in a number of ways:

Install the Microsoft core fonts with: sudo apt-get install ttf-mscorefonts-installer (this however did not work for me - not sure why)
Or copy the very similar font DejaVuSans.ttf to a file with the name arial.ttf in the directory /usr/share/fonts/truetype/dejavu
Or copy DejaVuSans.ttf to a file with the name arial.ttf in your object_detection directory (assuming you are running your code there)

[rishirm862]

Hello , how to display two label if image satisfy the two conditions .

[frederictost]

Hello,

I'm a bit stuck with GitHub, do I need to create a Branch ? How can I
submit a pull request?

Frédéric

Le sam. 11 nov. 2017 à 02:58, Austin Anderson [e-mail] a
écrit :

Thanks for the work you've done so far. Can you submit a pull request with
your change?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/models/issues/2771#issuecomment-343631608,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALrNiALlsvRFgE4cjyXe1dGALz8XzCZ9ks5s1P84gaJpZM4QaSQR
.

[tensorflowbutler]

Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

[mahemicro96]

With my new code I used the option font_size = 42
vis_util.visualize_boxes_and_labels_on_image_array(
image_np,
np.squeeze(boxes),
np.squeeze(classes).astype(np.int32),
np.squeeze(scores),
category_index,
use_normalized_coordinates=True,
line_thickness=0,
font_size = 42)
It works on Windows 8. I did not try on Linux.

If you don't set the font size it is automatically computed.

it also can visualize by 2 important threshold,min_score_thresh and line thickness

vis_util.visualize_boxes_and_labels_on_image_array(
image,
np.squeeze(boxes),
np.squeeze(classes).astype(np.int32),
np.squeeze(scores),
category_index,
use_normalized_coordinates=True,
line_thickness=0,
min_score_thresh=0.4)