Detr: nms parameters vs mAP

[kentaroy47]

First of all, thanks for presenting a great paper. It's one of the most innovative papers I've read recently in computer vision and sure many works will follow.

I was interested in the mAP performance with nms in Fig.4.
Do stronger nms (like nms=0.5) have similar mAP performance curves?
Maybe the mAP gets worse since more positive predictions will be deleted..

[fmassa]

Hi,

That's a very interesting question!

I evaluated one of the variants of the models we have (not exactly the one we have in the paper) with NMS=0.5, NMS=0.7 and no NMS.

Using NMS=0.7 dropped mAP compared to the baseline by 0.1 points, while using NMS=0.5 dropped mAP by 0.8 points

Here is a quick table summarizing it

| NMS | mAP delta | mAP small delta | mAP medium delta | mAP large delta
| --- | --- | --- | --- | --- |
No NMS | 0 | 0 | 0 | 0
0.7 | -0.1 | -0.1 | -0.2 | -0.3
0.5 | -0.8 | -0.3 | -0.7 | -1.0

So mAP decreases more for large objects.

I believe I've answered your question and as such I'm closing the issue, but let us know if you have further questions.

[kentaroy47]

Thanks for the fast answers! Yes, your experiments clarified my questions!
Thanks again for the great work.