Keras-retinanet: [proposal] drop either the compute-val-loss or the val-annotations argument

[AndreaPi]

In order to compute validation loss on a validation dataset, currently the train.py script requires that two arguments are passed:

• --compute-val-loss
• --val-annotations <VAL_ANNOTATIONS_DIR>

This is confusing, IMO, and it's not mentioned in the README:

https://github.com/fizyr/keras-retinanet/issues/976
https://github.com/fizyr/keras-retinanet/issues/1011

What about dropping one of the two? I think that having a single argument

--compute-val-loss <VAL_ANNOTATIONS_DIR>

would provide the same functionality, as well as a cleaner and more understandable API. Is this unfeasible for some reason?

[de-vri-es]

The validation set is also used to compute the mAP. So it is still useful without --compute-val-loss. There was some more discussion on that here: #884

[AndreaPi]

Thank you. I'm not sure I understand every point of that discussion, but if I'm not wrong, it says that -val-annotations <VAL_ANNOTATIONS_DIR> by itself (w/o --compute-val-loss) will still compute the mAP error on the validation set. Is this done for each epoch?

[AndreaPi]

The reason I ask is that I'm running a case with both --val-annotations <VAL_ANNOTATIONS_DIR> and--compute-val-loss right now, and Tensorboard shows two of each losses, but only one mAP, which I'm not sure whether it's computed on the train or the val set

[hgaiser]

It's poorly documented in the README because it "naturally" grew this way. The mAP is only computed on the validation set. Optionally, you can also compute the validation loss.

[AndreaPi]

ok, thanks @hgaiser . If I understand correctly, when I pass --val-annotations <VAL_ANNOTATIONS_DIR>, then the mAP is computed on the validation set, but since this computation, being based only on predictions, the code doesn't have to compute the validation loss too. In other words, we have two independent input parameters, because we have two independent computations. Is this correct?

[hgaiser]

Yes, exactly.