Ssd.pytorch: RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCTensorScatterGather

[qaz25323100]

I train the data from myself, and got the following error

Device: GTX 1080
Image size: 9053
Batch Size:32

 /data/RoadDamage/ssd/layers/modules/multibox_loss.py

 in forward(self, predictions, targets)
     98         print(conf_t.view(-1, 1).shape)
     99         print(batch_conf.shape)
--> 100         loss_c = log_sum_exp(batch_conf) - batch_conf.gather(1, conf_t.view(-1, 1))
    101 
    102         # Hard Negative Mining

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCTensorScatterGather.cu:67

conf_t      torch.Size([279424, 1])
batch_conf     torch.Size([279424, 8])

could anyone help me to solve the problem, thx

[tangdong1994]

change the match function in box_utils.py,if the background index label 0,
then conf = labels[best_truth_idx];
else if not background index, another label 0, conf = labels[best_truth_idx] +1;