Hello,
Has anyone got an error message at the beginning of the training claiming this:
Floating point exception(core dumped) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/faster_rcnn_end2end/solver.prototxt --weights data/imagenet_models/${NET}.v2.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/faster_rcnn_end2end.yml ${EXTRA_ARGS}
it obviously depends on my dataset but i cant figure what is the specific bug
thanks,
Did you ever find a solution? I have a similar problem
Hi guys, I also met this. It is caused by these lines of code in lib/rpn/proposal_layer.py
line 129: keep = _filter_boxes(proposals, min_size * im_info[2])
line 155: top[0].reshape(*(blob.shape))
min_size is 16 in default. When the image has large aspect ratio, all proposals are smaller than min_size * im_info[2], so keep is empty. Then line 155 reshape to empty will cause this.
So, what is the fix guys?
you can try to modify faster_rcnn_end2end.yml
set SCALES lower (ex:400),default value is 600
EXP_DIR: faster_rcnn_end2end
TRAIN:
HAS_RPN: True
IMS_PER_BATCH: 1
BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
RPN_POSITIVE_OVERLAP: 0.7
RPN_BATCHSIZE: 256
PROPOSAL_METHOD: gt
BG_THRESH_LO: 0.0
SCALES: [400]
TEST:
HAS_RPN: True
SCALES: [400]
Hi,have you solved the problem? I meet the same problem.
hey @yanxp and all,
sorry but it was a long while ago.
as i recall from my docs, it looks like this floating point issue is related to rois which r "too close" to image's bounds.
cant tell whats "too close" means numerically, but u can try modify your data by assigning some small margins between rois and image's bounds and see what margin figure will satisfy here
Most helpful comment
Hi guys, I also met this. It is caused by these lines of code in lib/rpn/proposal_layer.py
line 129:
keep = _filter_boxes(proposals, min_size * im_info[2])line 155:
top[0].reshape(*(blob.shape))min_sizeis 16 in default. When the image has large aspect ratio, all proposals are smaller thanmin_size * im_info[2], sokeepis empty. Then line 155 reshape to empty will cause this.