Faster-rcnn.pytorch: How to train on res18?

[EldorIbragimov]

I am trying to train on resnet 18. I download pretrained model. But I got this error:

Loading pretrained weights from /home/e/.torch/models/resnet18-5c106cde.pth
Traceback (most recent call last):
File "/home/e/faster-rcnn.pytorch/trainval_net.py", line 325, in
rois_label = fasterRCNN(im_data, im_info, gt_boxes, num_boxes)
File "/home/e/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
result = self.forward(input, *kwargs)
File "/home/e/faster-rcnn.pytorch/lib/model/faster_rcnn/faster_rcnn.py", line 50, in forward
rois, rpn_loss_cls, rpn_loss_bbox = self.RCNN_rpn(base_feat, im_info, gt_boxes, num_boxes)
File "/home/e/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
result = self.forward(input, *kwargs)
File "/home/e/faster-rcnn.pytorch/lib/model/rpn/rpn.py", line 62, in forward
rpn_conv1 = F.relu(self.RPN_Conv(base_feat), inplace=True)
File "/home/e/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
result = self.forward(input, *kwargs)
File "/home/e/.local/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 277, in forward
self.padding, self.dilation, self.groups)
File "/home/e/.local/lib/python2.7/site-packages/torch/nn/functional.py", line 90, in conv2d
return f(input, weight, bias)
RuntimeError: Given groups=1, weight[512, 1024, 3, 3], so expected input[1, 256, 27, 25] to have 1024 channels, but got 256 channels instead

[Rizhiy]

You need to adjust number of filters used in rpn and RCNN_head_to_tail since resnet18 and resnet50+ have different number of filters in resnet4.

[Worulz]

Hi @Rizhiy, I've tried adjusting the number of filters, but I'm not sure if I'm doing this right.

I've modified lines
In resnet.py
I changed self.dout_base_model to 256
num_layers changed to 18
... def __init__(self, classes, num_layers=101, pretrained=False, class_agnostic=False): self.model_path = 'data/pretrained_model/resnet101_caffe.pth' self.dout_base_model = 1024 ...

In rpn.py
I changed filters to 256
.. self.RPN_Conv = nn.Conv2d(self.din, 512, 3, 1, 1, bias=True) ... self.nc_score_out = len(self.anchor_scales) * len(self.anchor_ratios) * 2 # 2(bg/fg) * 9 (anchors) self.RPN_cls_score = nn.Conv2d(512, self.nc_score_out, 1, 1, 0) ... self.RPN_bbox_pred = nn.Conv2d(512, self.nc_bbox_out, 1, 1, 0) ...

how should RCNN_head_to_tail be configured?

When i try executing the script, I result with an error at this line
bbox_pred = self.RCNN_bbox_pred(pooled_feat)
in faster_rcnn.py.

[Rizhiy]

You shouldn't be changing the resnet architecture, as then you won't be able to load pretrained weights from the official repository. Instead, change the number of weights in the RPN layers and the first layer of RCNN_head_to_tail.

[JiasiWang]

Hi, I trained the faster rcnn on res18 with pascal voc 07, the mAP is about 55%. The performance is not as good as I thought, I wonder whether the result is normal. Thanks

[Worulz]

@JiasiWang what changes did you make to the RPN layer?. I'm still confused on this change. I've made the right changes to the resnet layers but I can't seem to know which lines specifically to change in the rpn layers.

I'm assuming it's somewhere between here?

https://github.com/jwyang/faster-rcnn.pytorch/blob/baa0385c0ec8cbd56fa204b1eea6aa15fe9ff0ea/lib/model/rpn/rpn.py#L27-L42

[kentaroy47]

@Worulz
Faced the same issue.
I changed the resnet.py to:

```resnet.py
class resnet(_fasterRCNN):
def __init__(self, classes, num_layers=101, pretrained=False, class_agnostic=False):
self.dout_base_model = 1024
if num_layers==101:
self.model_path = 'data/pretrained_model/resnet101_caffe.pth'
elif num_layers==50:
self.model_path = 'data/pretrained_model/resnet50-19c8e357.pth'
elif num_layers==34:
self.model_path = 'data/pretrained_model/resnet34-333f7ec4.pth'
self.dout_base_model = 256

lines(257)

if self.num_layers < 40:
    self.RCNN_cls_score = nn.Linear(512, self.n_classes)
    if self.class_agnostic:
      self.RCNN_bbox_pred = nn.Linear(512, 4)
    else:
      self.RCNN_bbox_pred = nn.Linear(512, 4 * self.n_classes)
else:
    self.RCNN_cls_score = nn.Linear(2048, self.n_classes)
    if self.class_agnostic:
      self.RCNN_bbox_pred = nn.Linear(2048, 4)
    else:
      self.RCNN_bbox_pred = nn.Linear(2048, 4 * self.n_classes)

```

[Worulz]

@kentaroy47 great thanks!. I'll test this soon.