Caffe: About detection_eval

[songjmcn]

When I test the SSD，the test output detection_eval。What‘s the meaning about detection_eval？How can I adjusting parameter with detection_eval？When I train the Net，the value of detection_eval is rasing，this is right？

[weiliu89]

detection_eval is mAP (mean Average Precision) used in object detection.

[siddharthm83]

@weiliu89 : couple of questions on results from your paper.

1. When your code calculates mAP (detection_eval), do you threshold detection (say at some confidence value) to be considered for mAP calculation?
2. For SSD300, is your reported mAP in paper/code (72.1 %) with test image size of 300x300 (for SSD300) or is it entire original image?
   I am getting 74.1% mAP (at IoU=0.5) with SSD300 trained model when I pass the full image for testing (not cropping to 300x300) and no thresholding of detections with my mAP script. Trying to find where the discrepancy is (if it's my calculation or some conditions are different during testing).

[weiliu89]

@siddharthm83

1. Yes. I only keep detections with score higher than 0.01. This is to balance the speed (of nms) and mAP. Setting such a low threshold can filter out most detections while the mAP is more or less same (at least it is true in my trials)
2. I warp the original test image to 300 x 300 for test. What do you mean using entire original image? Do you keep the aspect ratio of the image?

If you use original (==> higher resolution) image, it can possibly have better results although not optimal since you train your model on 300 x 300. I don't think 0.01 is a big issue here. You can try to play with the threshold to confirm.

[miladf2]

@siddharthm83 @songjmcn How did you guys use detection_eval? I am trying to calculate mAP for VOC dataset and my my own dataset?

[macqueen09]

@weiliu89 In a system which MobileNet , is this caffe , the detection_eval is bigger than 1 sometimes. What's that mean?

I1214 00:20:21.365911 19452 caffe.cpp:320] Batch 1, detection_eval = 1
I1214 00:20:21.365914 19452 caffe.cpp:320] Batch 1, detection_eval = 0
I1214 00:20:21.365918 19452 caffe.cpp:320] Batch 1, detection_eval = 1
I1214 00:20:21.365922 19452 caffe.cpp:320] Batch 1, detection_eval = 0
I1214 00:20:21.365926 19452 caffe.cpp:320] Batch 1, detection_eval = 2

and

I1214 03:03:31.287315 19452 caffe.cpp:337] detection_eval = 0.18
I1214 03:03:31.287320 19452 caffe.cpp:337] detection_eval = 0
I1214 03:03:31.287324 19452 caffe.cpp:337] detection_eval = 3.54
I1214 03:03:31.287329 19452 caffe.cpp:337] detection_eval = 0.18
I1214 03:03:31.287334 19452 caffe.cpp:337] detection_eval = 0
I1214 03:03:31.287338 19452 caffe.cpp:337] detection_eval = 0.18

and how can I get the mAP in detection system?
thanks very much