Maskrcnn-benchmark: can add tensorboardx for visualize the training status?

[mlinxiang]

🚀 Feature

current training information only display in console, this is not very anschauung，can add tensorboardx to support loss or feature map 's visualize

[fmassa]

Adding support for TensorBoard would be a good addition, but I'm currently without bandwidth to add it.

Another alternative is to have a script to parse the logs that are printed, and plot them in a jupyter notebook for example (that's what I've been doing).

[steve-goley]

I'm planning to add a visdom plotter. Also the only feature I miss from TF OD API is the concurrent eval loop. That's also on my list.

[fmassa]

@steve-goley what do you mean by concurrent eval loop?
Currently, it is possible to evaluate using multiple GPUs, is that what you meant?

[steve-goley]

In TF OD API the user can run an eval process that

1. Loads the last checkpoint and evaluates validation images (or a subset of them)
2. Computes AP for the classes
3. Plots AP in Tensorboard
4. Visualizes images in Tensorboard
5. Waits until a new checkpoint is created and repeats the process

It requires that you keep one GPU for eval but it's nice to have some sort of visibility while training. FRCNN loss outputs are hard to use as a gauge for performance plateaus, IMHO, AP over the validation set is much more informative.

[fmassa]

@steve-goley oh, I see, yes, that would definitely be a plus to have!

[mlinxiang]

@steve-goley can provide any docs describe TF OD API's visualize support mechanism，so may i can add some doc to do this

[steve-goley]

@mlinxiang TensorFlow uses TensorBoard. I haven't dug into the checkpoint structure, yet. I think that a similar capability could be constructed with visdom. There are some advantages and disadvantages to visdom over Tensorboard / tensorboardX. I've found visdom quite easy to use and the API, much like many things FAIR, is quite readable and concise. @fmassa do you have a strong opinion on tensorboardX vs. visdom.

[soumith]

given the rich nature and potential of mask-rcnn visualizations, where a lot of custom visualizations might make sense, Visdom seems better longer-term. Visdom has trivial ways to construct rich custom visualizations.

[mlinxiang]

has visdom integrate into pytorch?

[fmassa]

I think Visdom works with PyTorch

[wangg12]

I think using tensorboardX to visualize the accuracies and losses in real time is a good idea.