Pytorch-yolov3: Why ignore anchors with iou bigger than threshold?

[jjjjohnson]

https://github.com/eriklindernoren/PyTorch-YOLOv3/blob/959e0ff43f5b82bdacef87f4240bae8415eac45b/utils/utils.py#L181

[ydixon]

The idea is not to train for any "noobjectess" if there is a object around there which is determined by "anch_ious > ignore_thres". But everyone seems to have their own way of doing it, since there's isn't a lot of discussion around. In this repo, he did it this way. In my repo, i did it with

            #preds - [bs x A x W x H x (5+C)]  
            tmp_gt_boxes = torch.FloatTensor([gx, gy, gw, gh])
            tmp_pred_boxes = preds[b].view(-1, 4)
            tmp_ious, _ = torch.max(bbox_iou(tmp_pred_boxes, tmp_gt_boxes, mode="cxcywh"), 1)
            ignore_idx = (tmp_ious > ignore_thres).view(nA, nH, nW)
            noobj_mask[b][ignore_idx] = 0

            #find best fit anchor for each ground truth box
            tmp_gt_boxes = torch.FloatTensor([[0, 0, gw, gh]])
            tmp_anchor_boxes = torch.cat((torch.zeros(nA, 2), anchors), 1)
            tmp_ious = bbox_iou(tmp_anchor_boxes, tmp_gt_boxes, mode="cxcywh")
            best_anchor = torch.argmax(tmp_ious, 0).item()

            #find iou for best fit anchor prediction box against the ground truth box
            tmp_gt_box = torch.FloatTensor([gx, gy, gw, gh]).unsqueeze(0)
            tmp_pred_box = preds[b, best_anchor, gj, gi].view(-1, 4)
            tmp_iou = bbox_iou(tmp_gt_box, tmp_pred_box, mode="cxcywh")`

I haven't read the C implementation of the original Yolo yet. Hope someone more educated can shed some light.

[jjjjohnson]

Hi @ydixon ,
I agree with you. Here conf_mask is initialized by
conf_mask = torch.ones(nB, nA, dim, dim)
conf_mask is essentially noobj_mask. The confusing part is at https://github.com/eriklindernoren/PyTorch-YOLOv3/blob/959e0ff43f5b82bdacef87f4240bae8415eac45b/utils/utils.py#L190

where the best iou anchor at that cell is set back to 1. That should be the cell which has the object with best iou. Right?

[ydixon]

Which is why I did it different. In this repo, there is only one mask that decides whether to train for objectness (including both obj and noobj). The idea of setting the mask is:

1. First set the entire mask to 1s (meaning include every spot)
2. Find place/area of pixels that has object, do not train for objectness (noobj) , " conf_mask[b, anch_ious > ignore_thres] = 0 "
3. Find the anchor box that has best iou with ground truth box dimension wise, ignore location
4. Set conf_mask[b, best_n, gj, gi] = 1 would mean train for objectness(obj) for that particular best anchor box at that specific location where the ground truth box is located.

[jjjjohnson]

Thanks @ydixon !
but I think the shape of conf_mask is (nB, nA, dim, dim) and conf_mask[b, anch_ious > ignore_thres] = 0 sets all the numbers in some of nA channels to zero. Which is not the same as you stated in point 2:
"Find place/area of pixels that has object, do not train for objectness (noobj) , " conf_mask[b, anch_ious > ignore_thres] = 0 ""

[ydixon]

Yes, I agree. That's why in my own implementation I did it different. I haven't run through all the epochs to confirm if it converges to good model but I suspect it does. Maybe there's a reason to do this, but only the author would know.

[bobo0810]

@jjjjohnson

Two lines of code need to be matched with understanding.

conf_mask[b, anch_ious > ignore_thres] = 0

This means that the grid responsible for predicting the object and the grid around it are set to zero. Indicates that they are not involved in training.

conf_mask[b, best_n, gj, gi] = 1

This means that the grid responsible for predicting the object is set to 1 again.

Thought: Similar to NMS. For an object, filter out the grid with a higher overlap rate, leaving only the grid of overlap rates.

[jjjjohnson]

Hi @bobo0810
Note that conf_mask = torch.ones(nB, nA, dim, dim). Does it mean all the (nA, dim, dim) where anch_ious <= ignore_thres are responsible for predicting the object?

[bobo0810]

@jjjjohnson
https://github.com/bobo0810/PyTorch-YOLOv3-master
The repository is a version that adds a lot of annotate to make it easier to understand（Chinese）.

As for the question you asked, I am thinking now.

[ydixon]

I already checked the values in debugger, it's wiping all the mask across all grids. There is a problem here.

[WormCoder]

@ydixon
I read the c code of origin yolo. iou between predicted box and truth box is calculated, and then compared with ignore_thres, just like your implementation. However, according to the yolo v3 paper, iou between anchor and truth should be calculated here, just like @bobo0810 's implementation.
Maybe both are OK.

[ydixon]

@WormCoder I haven't had time to study the C code yet. Thanks for confirming. As for the right way to do it, I'm not sure too. I'm still experimenting with the loss function. Has anyone has got this repo's model to converge? ( I don't have the GPU resource :p)

[huw]

@ydixon I have a slight modification of this repo converging, and having gone through the Darknet code, both appear to be doing a similar thing. The relevant code is here: https://github.com/pjreddie/darknet/blob/61c9d02ec461e30d55762ec7669d6a1d3c356fb2/src/yolo_layer.c#L178

In this case, Joseph Redmon initially sets the gradient to 0 - l.output[obj_index], which effectively says "this box should have a confidence of zero". Then if the box overlaps enough with the anchor (best_iou > l.ignore_thresh), he sets the gradient to 0, which effectively says "ignore this box completely". But if the box overlaps a LOT (best_iou > l.truth_thresh), then he sets it to 1 - l.output[obj_index], which effectively says "this box should have a confidence of 1".

This repo is doing a slightly different thing, because it always sets the box with the best IoU to 1, even if the IoU isn't that good. But I suspect that these differences are ultimately dealt with at the non-maximum suppression stage so it doesn't particularly matter — this repo definitely converges with some small modifications and bug fixes.

[eriklindernoren]

Fixed in the current version.