Vision: Mask R-CNN: Dictionary of losses for the validation set

[ibtingzon]

Based on mask_rcnn.py:
https://github.com/pytorch/vision/blob/master/torchvision/models/detection/mask_rcnn.py
it seems like you can track the training loss by setting model.train() and getting the dictionary of losses: loss_dict = model(images, targets).

However, I'm interested in getting the validation loss.
When I set model.eval(), instead of returning a dictionary of losses, it returns the actual predictions.

How can I get the dictionary of losses for my validation set, given that I set model.eval()? Any help would be much appreciated.

[fmassa]

You would need to modify the implementation to always compute the predictions, even during training.

Basically, you'll need to remove the else and instead always compute the predictions, so unindent this block
https://github.com/pytorch/vision/blob/367e8514c5d5e8528330081f25efeb17dd5e7ebd/torchvision/models/detection/roi_heads.py#L550-L560

If you want to do the same for Mask R-CNN (or Keypoint R-CNN), you'll need to also need to do a similar thing.

Then you'll need to modify https://github.com/pytorch/vision/blob/367e8514c5d5e8528330081f25efeb17dd5e7ebd/torchvision/models/detection/generalized_rcnn.py#L59-L60 to also return the detections.

Let me know if you have more questions.

[ibtingzon]

Great! If I wanted the model to return both detections and losses during both train and eval mode, would I need to comment out if self.training and unindent this part of the code:

https://github.com/pytorch/vision/blob/367e8514c5d5e8528330081f25efeb17dd5e7ebd/torchvision/models/detection/roi_heads.py#L564-L572

[fmassa]

This part of the code is for masks, but yeah, you would need to remove those conditions