Py-faster-rcnn: Detection/training for small objects (Question)

[paulomarcos]

TLDR: How can I detect small objects using py-faster-rcnn? Which parameters should I change when running the demo or for training?

I have successfully trained using the ImageNet dataset. It runs fine and it detects the objects as expected. However, when the objects in the scene are small, the detection doesn't work. The next thing I have done is to train the dataset using the ImageNet format but with my own images and annotations, adding a new class to the previous list, which is of different types of cups. The train script runs fine, it gives the output as expected, but in the demo when I add the new (7th) class to the list, it gives the following error:

Detection took 0.052s for 300 object proposals
Traceback (most recent call last):
  File "./tools/demo.py", line 201, in <module>
    demo(net, im)
  File "./tools/demo.py", line 117, in demo
    cls_scores = scores[:, cls_ind]
IndexError: index 6 is out of bounds for axis 1 with size 6

To be more precise, I would like to detect the cup on this image (original size 1920x1080):

If the cup is on the table and I crop and resize the frame with OpenCV to the resulting image below, the algorithm detects it, but not as the new class that I have trained.

So I would like to know what I am doing wrong and how can I fix it.
Thank you.

[harjatinsingh]

Could you find an answer to make faster rcnn work for smaller objects in images ?

[paulomarcos]

@harjatinsingh So far I havent being able to successfully make it work for smaller images as I wanted. However, it seems changing the values of the ratios in generate_anchors.py does make the algorithm to recognize smaller objects, but the bounding box looses precision. For instance, what I have done is changing the code below from this:
def generate_anchors(base_size=16, ratios=[0.5, 1, 2], scales=2**np.arange(3, 6)):
To this:
def generate_anchors(base_size=16, ratios=[0.3, 0.75, 1], scales=2**np.arange(3, 6)):

Also, it seems that changing the values of anchors does work as noted in #161 but I couldnt make it work for me. So my question (in both issues) is still pending.

[harjatinsingh]

Thanks for the assistance. Please update on this thread if you make it work.

[hito0512]

@harjatinsingh Could you find an answer to make faster rcnn work for smaller objects in images ?

[ntajbakhsh]

I remember py-faster-rcnn would remove detections smaller than 16 pixels. Try to disable that. You may also get higher sensitivity by turning off NMS or by making it less aggressive. More sophisticated approaches can be to use dilated convolution or deconvolution layers.

[Frank-Ge]

Just like @ntajbakhsh said, py-faster-rcnn would remove detections smaller than a specific size, and the function is _filter_boxes(boxes, min_size) in py-faster-rcnn/lib/rpn proposal_layer.py. I changed the min_size and it can detect small objects.