Centernet: Why are the dimensions of predictions and ground truth different?

[ljjyxz123]

@xingyizhou
I printed out the dimensions of the predictions from CenterNet, and the ground truth. But I found they are different.

The code I wrote is:
(1) For ground truth:
`

    for k in batch:

    batch[k] = batch[k].to(device=opt.device, non_blocking=True)

    print('batch: ', k, batch[k].size())

`

The printed contents are shown as below:

(2)For predictions:
`

  print('output[wh].size(): ',  output['wh'].size())

  print('output[reg].size(): ', output['reg'].size())

  print('output[hm].size(): ', output['hm'].size())

`

The printed contents are shown as below:

I think the predicted results are right in dimensions, because there are two 128s. But for the ground truth, why the dimensions of 'wh' and 'reg' are [32, 128, 2], here 32 is the batch_size.

I thought the ground truth dimensions of 'wh' and 'reg' should be [batch_size, 128, 128, 2], or [batch_size, 2, 128, 128], which is exactly same with the predictions.

Do you know why this happened?
Thank you!

[xingyizhou]

This is a good question and is indeed counter-intuitive. The network predictions are in shape Batch x C x H x W. For all the attributes prediction (e.g., wh, reg, depth ...), we only add supervision at the peak locations. To implement this, we use the gather function (transpose_and_gather_feature in our code) to exact the peak values in the predicted Batch x C x H x W tensor to Batch x M x C tensor, where M = 128 is the max number of peaks in the image. We need to use a fixed number of objects (M) because we need a constant shape to form a batch. In most images there are <128 peaks, we multiply 0 for the rest locations when calculating the loss.

[ljjyxz123]

This is a good question and is indeed counter-intuitive. The network predictions are in shape Batch x C x H x W. For all the attributes prediction (e.g., wh, reg, depth ...), we only add supervision at the peak locations. To implement this, we use the gather function (transpose_and_gather_feature in our code) to exact the peak values in the predicted Batch x C x H x W tensor to Batch x M x C tensor, where M = 128 is the max number of peaks in the image. We need to use a fixed number of objects (M) because we need a constant shape to form a batch. In most images there are <128 peaks, we multiply 0 for the rest locations when calculating the loss.

@xingyizhou
Thank you very much!
I get it~