Py-faster-rcnn: proposal_layer.py error

[aaronjie]

I configured py-faster-rcnn with cpu, but get following error:

Demo for data/demo/000456.jpg
Traceback (most recent call last):
File "demo.py", line 155, in
demo(net, im_name)
File "demo.py", line 84, in demo
scores, boxes = im_detect(net, im)
File "/home/aaron/Downloads/py-faster-rcnn/tools/../lib/fast_rcnn/test.py", line 154, in im_detect
blobs_out = net.forward(**forward_kwargs)
File "/home/aaron/Downloads/py-faster-rcnn/tools/../caffe-fast-rcnn/python/caffe/pycaffe.py", line 105, in _Net_forward
self._forward(start_ind, end_ind)
File "/home/aaron/Downloads/py-faster-rcnn/tools/../lib/rpn/proposal_layer.py", line 146, in forward
keep = keep[:post_nms_topN]
TypeError: 'NoneType' object has no attribute '__getitem__'

i'm a new learner of this, hope brothers to help me, thank you~~

[dkonomis]

My solution was to do:

if post_nms_topN > 0:
    if keep is None:
        keep = []
    else:
        keep = keep[:post_nms_topN]

instead of

if post_nms_topN > 0:
    keep = keep[:post_nms_topN]

However, after I made it run, I still get a floating point (core dumped) error.

[VishnuMadhu]

I faced this issue while making the cpu only build of py-faster-rcnn. The issue was due to a wrong nms return from the nms wrapper.

While editing nms_wrapper.py , make sure that the function returns cpu_nms(dets, thresh).

def nms(dets, thresh, force_cpu=False):
   #Dispatch to either CPU or GPU NMS implementations.

    if dets.shape[0] == 0:
        return []
    if cfg.USE_GPU_NMS and not force_cpu:
        #return gpu_nms(dets, thresh, device_id=cfg.GPU_ID)
        return cpu_nms(dets, thresh)
    else:
        return cpu_nms(dets, thresh)