Ssd.pytorch: Bug in augmentations.py

[pierluigiferrari]

In line 269 of augmentations.py it should be or instead of and:

# is min and max overlap constraint satisfied? if not try again
if overlap.min() < min_iou and max_iou < overlap.max():
    continue

In particular, for the default SSD settings where max_iou = 1.0, max_iou < overlap.max() will always evaluate to False, which means the expression above will never be True, and so the IoU bounds effectively don't matter at all.

[hao522]

agree, looks like a bug

[JingyuanHu]

Agree, it is a bug

[pierluigiferrari]

I believe there's another inaccuracy in the logic of RandomSampleCrop here:

# calculate IoU (jaccard overlap) b/t the cropped and gt boxes
overlap = jaccard_numpy(boxes, rect)

# is min and max overlap constraint satisfied? if not try again
if overlap.min() < min_iou and max_iou < overlap.max():
    continue

Even if the bug in my OP were fixed, this logic accepts a crop as valid only if all boxes meet the minimum Jaccard overlap with the sampled crop. The code of the original SSD implementation, however, accepts a crop as valid if at least one box meets the minimum Jaccard overlap:

  // Check constraints.
  bool found = false;
  for (int i = 0; i < object_bboxes.size(); ++i) {
    const NormalizedBBox& object_bbox = object_bboxes[i];
    // Test jaccard overlap.
    if (has_jaccard_overlap) {
      const float jaccard_overlap = JaccardOverlap(sampled_bbox, object_bbox);
      if (sample_constraint.has_min_jaccard_overlap() &&
          jaccard_overlap < sample_constraint.min_jaccard_overlap()) {
        continue;
      }
      if (sample_constraint.has_max_jaccard_overlap() &&
          jaccard_overlap > sample_constraint.max_jaccard_overlap()) {
        continue;
      }
      found = true;
    }

Any thoughts? If I am right, that would be a significant difference. For most images it's highly unlikely to get a crop in which all boxes meet a high Jaccard overlap, so I would assume that the vast majority of accepted random crops end up being for the minimum Jaccard overlap of 0.0 or by returning the unaltered input image, which kind of defeats the purpose of this whole sophisticated cropping logic.

[heinzermch]

Even if the bug in my OP were fixed, this logic accepts a crop as valid only if all boxes meet the minimum Jaccard overlap with the sampled crop.

I agree

Any thoughts? If I am right, that would be a significant difference. For most images it's highly unlikely to get a crop in which all boxes meet a high Jaccard overlap, so I would assume that the vast majority of accepted random crops end up being for the minimum Jaccard overlap of 0.0 or by returning the unaltered input image

Depends on your dataset, how many boxes you usually have per image, and how far they are spread out. But I agree that this significantly limits the possible crops, and hence the usefulness of the cropping algorithm.

I suggest as fix:

if overlap.max() < min_iou or overlap.min() > max_iou:
    continue

[pierluigiferrari]

Depends on your dataset, how many boxes you usually have per image, and how far they are spread out.

As soon as you have more than one object in a given crop, it becomes more or less impossible for all objects to have a high Jaccard overlap of 0.7 or 0.9 with the crop, because for this to be the case they would also have to have a high Jaccard overlap with each other.

But leaving aside the consequences of this cropping logic, what I'm driving at is that it is not the same as the cropping logic in the Caffe implementation.

[dongdonghy]

Agree!