Pytorch-yolov3: collate_fn has problem dealing with images having no boxes

[favrei]

https://github.com/eriklindernoren/PyTorch-YOLOv3/blob/47b7c912877ca69db35b8af3a38d6522681b3bb3/utils/datasets.py#L134-L140

This part of code tries to eliminate empty placeholder, i.e., images without any box, in the variable targets, then adding indices to all boxes.
If we have, say, 8 images in the batch, but only the 4th, to the 7th image have boxes.
The targets should be like this:
[[4, ...]
[4, ...]
[5, ...]
[6, ...]
[6, ...]
[7, ...]
[7, ...]]

But current implementation eliminates the empty placeholder before assigning the indices. The indices of boxes, in this case, will eventually be 0, ... , 3, not 4, ..., 7 what we anticipated.

This is my local edit FYR, it works fine at my side.

    def collate_fn(self, batch):
        paths, imgs, targets = list(zip(*batch))
        # Add sample index to targets
        for i, boxes in enumerate(targets):
            if boxes is None:
                continue
            boxes[:, 0] = i
        # Remove empty placeholder targets
        targets = [boxes for boxes in targets if boxes is not None]
        targets = torch.cat(targets, 0)

[mcombalia]

Also, there is a problem with

targets = torch.cat(targets, 0)

When the entire batch doesn't contain annotations.

[cjnjuwhy]

Also, there is a problem with

targets = torch.cat(targets, 0)

When the entire batch doesn't contain annotations.

this would be useful https://github.com/eriklindernoren/PyTorch-YOLOv3/pull/163/commits/83c3d8b2f440f78715fb674b3d318c63ffe3eb16

[ujsyehao]

@killer9619
If I follow original implementation,

The indices of boxes, in this case, will eventually be 0, ... , 3, not 4, ..., 7

0, ..., 3 just represents an index, Does it influence mAP? Can you give me some advice...

I understand it, gt box index change, but the model predicts boxes following original index, ex:
img1 -> label 1
img2 -> label 2
img3 -> label 3
img4 -> label 6(label 4 doesn't exist, so replace it use label6)

[soldier828]

https://github.com/eriklindernoren/PyTorch-YOLOv3/blob/47b7c912877ca69db35b8af3a38d6522681b3bb3/utils/datasets.py#L134-L140

This part of code tries to eliminate empty placeholder, i.e., images without any box, in the variable targets, then adding indices to all boxes.
If we have, say, 8 images in the batch, but only the 4th, to the 7th image have boxes.
The targets should be like this:
[[4, ...]
[4, ...]
[5, ...]
[6, ...]
[6, ...]
[7, ...]
[7, ...]]

But current implementation eliminates the empty placeholder before assigning the indices. The indices of boxes, in this case, will eventually be 0, ... , 3, not 4, ..., 7 what we anticipated.

This is my local edit FYR, it works fine at my side.

    def collate_fn(self, batch):
        paths, imgs, targets = list(zip(*batch))
        # Add sample index to targets
        for i, boxes in enumerate(targets):
            if boxes is None:
                continue
            boxes[:, 0] = i
        # Remove empty placeholder targets
        targets = [boxes for boxes in targets if boxes is not None]
        targets = torch.cat(targets, 0)

Thank U！
After fix, get mAP=0.5704@608 mAP=0.5145@416

[summeryumyee]

mark

[loopoook]

mAP=57.04@608
mAP=54.9@416