Centernet: [question] Is the input image to models expected to be in BGR format?

[HamsterHuey]

I don't see this specifically mentioned in the documentation files, but when I look through the code, it appears that after default cv2.imread, the only changes done are to transpose the image to the pytorch format of C x H x W. But it appears that the channel information itself is still in the default BGR format that open-cv uses.

Does this mean that all the model weights in the model zoo assume that the input image inp is in BGR format?

[xingyizhou]

Yes.

[HamsterHuey]

Thanks. On a related note, I notice that the mean and std values for the dataset are taken from CornerNet. Do you know if those are COCO specific statistics? I would have thought that they might be Imagenet statistics since all of these models are typically trained starting with ImageNet and then transferring to COCO, but the numbers don't match ImageNet statistics. I also couldn't find any published dataset statistics for COCO.

[y0un5]

@HamsterHuey
It seems to be COCO specific statistics(in BGR order). I couldn't find any reference, so computed the statistics with following code. I computed on only COCO val2017, but got almost same values with those in coco.py.

COCO statistics computation code and result

import os

import cv2 
import numpy as np

from tqdm import tqdm

image_dir = 'data/coco/val2017'

Bs = []
Gs = []
Rs = []
pixel_counts = []

for filename in tqdm(os.listdir(image_dir)):
  im = cv2.imread(os.path.join(image_dir, filename)) # BGR
  height, width, _ = im.shape

  Bs.append(np.sum(im[:, :, 0]) / 255)
  Gs.append(np.sum(im[:, :, 1]) / 255)
  Rs.append(np.sum(im[:, :, 2]) / 255)

  pixel_counts.append(height * width)

total_pixel_num = np.sum(pixel_counts)
B_mean = np.sum(Bs) / total_pixel_num
G_mean = np.sum(Gs) / total_pixel_num
R_mean = np.sum(Rs) / total_pixel_num
print(B_mean)
print(G_mean)
print(R_mean)



B_squared_mean_diffs = []
G_squared_mean_diffs = []
R_squared_mean_diffs = []

for filename in tqdm(os.listdir(image_dir)):
  im = cv2.imread(os.path.join(image_dir, filename)) # BGR

  B_squared_mean_diffs.append(np.sum(np.square(im[:, :, 0] / 255 - B_mean)))
  G_squared_mean_diffs.append(np.sum(np.square(im[:, :, 1] / 255 - G_mean)))
  R_squared_mean_diffs.append(np.sum(np.square(im[:, :, 2] / 255 - R_mean)))

B_std = np.sqrt(np.sum(B_squared_mean_diffs) / total_pixel_num)
G_std = np.sqrt(np.sum(G_squared_mean_diffs) / total_pixel_num)
R_std = np.sqrt(np.sum(R_squared_mean_diffs) / total_pixel_num)                                                                                                                                                                                
print(B_std)
print(G_std)
print(R_std)

0.406712961305471
0.446107718309691
0.470023912393692

0.2882194077069855
0.2744769722240565
0.27934358115807123

@xingyizhou
Pascal statistics in pascal.py uses RGB order. It is imagenet statistics in RGB order(see).

  mean = np.array([0.485, 0.456, 0.406],
                   dtype=np.float32).reshape(1, 1, 3)
  std  = np.array([0.229, 0.224, 0.225],
                   dtype=np.float32).reshape(1, 1, 3)

So, when use Pascal dataset, channel orders are different between image and mean/std statistics. I think there might be some negative effect. What do you think?

[HamsterHuey]

Thanks @y0un5 . That's good to know. You are correct that the pascal ordering seems wrong. In general, all the dataset stats should be quite similar, which does seem to be the case when comparing BGR ImageNet vs BGR COCO stats. Even the BGR Pascal stats above are quite similar.

I'm curious if fixing the Pascal stats would actually improve performance of the model. In general, the wrong ordering will simply make the channels slightly positively or negatively biased. Given that we are using Relu + Batchnorm in most architectures, I'm not sure if this will end up mattering too much as the networks will probably be able to account for this. But I do agree that ideally those stats for Pascal VOC should be switched around to BGR ordering.