Maskrcnn-benchmark: how to visualize a detection model?

[veraposeidon]

❓ Questions and Help

I was trying to visualize a detection model built in maskrcnn_benchmark, for deeper analytics.
I have tried two ways:

1. export to onnx format model, then use Netron for visualize.
2. use pytorchviz for visualize.

But they all raise a error like Auto nesting doesn't know how to process an input object of type maskrcnn_benchmark.structures.image_list.ImageList. Accepted types: Tensors, or lists/tuples of them

Obviously, there are two mountains stopped me, ImageList and BoxList, it seems like I should wait until ONNX standard supports these operators.

I am here looking for some help.
Is there any solutions or tricks to visualize a model?

[fmassa]

I haven't tried visualizing the model myself, but I'd expect that by applying some patches from https://github.com/facebookresearch/maskrcnn-benchmark/pull/138 you will be able to export the model to a PyTorch JIT graph. But I'm not sure there are tools available for visualizing them.

[qizhuli]

Hi @fmassa, I have enjoyed using maskrcnn-benchmark and would like to tweak the architecture of some of the networks implemented here for my research.

However, from this issue, it seems that there is currently no quick and easy way to visualise a model.

I'd be extremely grateful if you could shed some light on any practical workaround to check what has been coded up is indeed what has been intended, in terms of the network architecture. Many thanks!

[fmassa]

@qizhuli You can probably use https://github.com/szagoruyko/pytorchviz for that, after a few tweaks here and there

[qizhuli]

Thanks @fmassa! I managed to visualise maskrcnn-benchmark models using pytorchviz as you suggested. Thought I would share the script in case it might help others :)

[fmassa]

@qizhuli nice!
Can you share what the visualization looks like for one of the models?

[qizhuli]

Sure! Here is the output of my script for ResNet50-FPN Mask-RCNN network ($root/configs/e2e_mask_rcnn_R_50_FPN_1x.yaml). Simply save it at $file_path and run xdot $file_path to visualise it.

Edit: I rendered it into a pdf file for your viewing convenience :)

[veraposeidon]

@qizhuli It works, thanks a lot!
And instead of using FileIO to save a graph infomation, I found a more convenient way to do this.
Just use:

graph.format = format
graph.reder(filename)

where:

• graph is graph representation returned by funtion make_dot.
• 'format' is the file format you want to save. For your infomation, here is a list of formats that graphviz support. https://www.graphviz.org/doc/info/output.html
• 'filename' is the path you want to save.

for example.
Instead of:

graph = make_dot(loss, params=dict(list(model.named_parameters()) + [('dummy_image',dummy_image)]))
with open(args.output_path, 'w') as f:
    f.write(str(graph))
f.close()

Use:

graph = make_dot(loss, params=dict(list(model.named_parameters()) + [('dummy_image',dummy_image)]))
graph.format = 'pdf'
graph.render("visPDF")

then, this will generate two files: visPDF and visPDF.pdf. visPDF is text file same as FileIO generated, and visPDF.pdf is the visualized file.

by the way, do find a good format after tries. png format generated blurred image, and jpg report bugs. pdf works fine.

[qizhuli]

@veraposeidon Thanks for your comments! Indeed it's cleaner to call graph.render instead of invoking the FileIO APIs. I have revised the gist accordingly and added a new optional input argument --output_format with which you can instruct the script to render the graph in a specific format. The default is set to 'pdf'.

[PyZhangBIT]

@qizhuli Hi! How can i visualize a RetinaNet, such as configs/retinanet/retinanet_R-101-FPN_1x.yaml ?
When i replace the configuration file, it failed with KeyError: 140667121782144!