Models: Display the object ground truth box in tensorboard

[mbenami]

Hi
Is it possible to display the objects ground truth along with the detection in tensorboard ?
I'm using the object detection models
Thnaks

Please go to Stack Overflow for help and support:

http://stackoverflow.com/questions/tagged/tensorflow

Also, please understand that many of the models included in this repository are experimental and research-style code. If you open a GitHub issue, here is our policy:

1. It must be a bug or a feature request.
2. The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

---

System information

• What is the top-level directory of the model you are using:
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
• TensorFlow installed from (source or binary):
• TensorFlow version (use command below):
• Bazel version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:
• Exact command to reproduce:

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

[cipri-tom]

Yes, it is possible, if you set visualization_export_dir in the eval_config.
If the directory doesn't exist, the images won't be saved, but it will still be displayed in tensorboard.

Let's use StackOverflow for further support questions, shall we ? 😃

[mbenami]

Hi @cipri-tom thanks for the replay...

I try to add this option to my network.config file:

eval_config {
    visualization_export_dir: "/home/ubuntu/models-tf/research/notexist"
...}

but got an error when tensorflow try to save the image

File "/home/ubuntu/models-tf/research/object_detection/utils/visualization_utils.py", line 70, in save_image_array_as_png
image_pil.save(fid, 'PNG')
File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1928, in save
save_handler(self, fp, filename)
File "/usr/local/lib/python2.7/dist-packages/PIL/PngImagePlugin.py", line 709, in _save
fp.write(_MAGIC)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 101, in write
self._prewrite_check()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 87, in _prewrite_check
compat.as_bytes(self.__name), compat.as_bytes(self.__mode), status)
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.NotFoundError: /home/ubuntu/models-tf/research/notexist/export-image-0.png

I succeed to work this by surrounding the save with try and except
in object_detection/utils/visualization_utils.py line 70

try:  
    image_pil.save(fid, 'PNG') 
except:
    pass 

(not sure if this is the right place for this fix)

Then I got the ground truth display but a just black box with no reference to the class it represents
as it says:

def visualize_boxes_and_labels_on_image_array(...
...
scores: a numpy array of shape [N] or None. If scores=None, then
this function assumes that the boxes to be plotted are ground truth
boxes and plot all boxes as black with class and score GT. '

To work around that, I change in
eval_util.py line 242:

vis_utils.visualize_boxes_and_labels_on_image_array(
    image,
    groundtruth_boxes,
    groundtruth_classes,
    None,

to pass the class names
and in visualization_utils.py:
line 173:

text_color = 'balck'  
if color == 'black':
    text_color = 'white'
draw.text(
    (left + margin, text_bottom - text_height - margin),
    display_str,
    fill='black',
    fill=text_color,
    font=font)

line 385:

if classes is not None:
    if classes[i] in category_index.keys():
        class_name = category_index[classes[i]]['name']
     else:
         class_name = 'N/A'
     display_str = '{}: {}'.format(class_name,'GT')
     box_to_display_str_map[box].append(display_str)

[skye]

@cipri-tom is right, please move open a question on StackOverflow for usage-related questions.