Faster-rcnn.pytorch: Cannot run Visual Genome VGG16 model

[GPaolo]

Whenever I try to run VGG16 trained on Visual Genome, I get errors of dimensions mismatches between the models.

I use this command to launch the test:
python demo.py --net vgg16 --dataset vg --checksession 1 --checkepoch 19 --checkpoint 48611 --cuda --load_dir ./data/pretrained_model
but I obtain the following error:

Traceback (most recent call last): File "demo.py", line 197, in <module> fasterRCNN.load_state_dict(checkpoint['model']) File "/root/faster-rcnn.pytorch/.venv/lib/python3.5/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for vgg16: While copying the parameter named "RCNN_rpn.RPN_cls_score.weight", whose dimensions in the model are torch.Size([18, 512, 1, 1]) and whose di mensions in the checkpoint are torch.Size([24, 512, 1, 1]). While copying the parameter named "RCNN_rpn.RPN_cls_score.bias", whose dimensions in the model are torch.Size([18]) and whose dimensions in t he checkpoint are torch.Size([24]). While copying the parameter named "RCNN_rpn.RPN_bbox_pred.weight", whose dimensions in the model are torch.Size([36, 512, 1, 1]) and whose di mensions in the checkpoint are torch.Size([48, 512, 1, 1]). While copying the parameter named "RCNN_rpn.RPN_bbox_pred.bias", whose dimensions in the model are torch.Size([36]) and whose dimensions in t he checkpoint are torch.Size([48]). While copying the parameter named "RCNN_cls_score.weight", whose dimensions in the model are torch.Size([21, 4096]) and whose dimensions in t he checkpoint are torch.Size([2501, 4096]). While copying the parameter named "RCNN_cls_score.bias", whose dimensions in the model are torch.Size([21]) and whose dimensions in the check point are torch.Size([2501]). While copying the parameter named "RCNN_bbox_pred.weight", whose dimensions in the model are torch.Size([84, 4096]) and whose dimensions in t he checkpoint are torch.Size([10004, 4096]). While copying the parameter named "RCNN_bbox_pred.bias", whose dimensions in the model are torch.Size([84]) and whose dimensions in the check point are torch.Size([10004]).

Am I missing something?

[CyanideCentral]

I encountered basically the same problem today. Have you fixed it?

My output:
load checkpoint /srv/share/jyang375/models/vgg16/coco/faster_rcnn_1_5_25799.pth
Traceback (most recent call last):
File "demo.py", line 194, in
fasterRCNN.load_state_dict(checkpoint['model'])
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 721, in load_state_dict
self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for vgg16:
While copying the parameter named "RCNN_rpn.RPN_cls_score.bias", whose dimensions in the model are torch.Size([18]) and whose dimensions in the checkpoint are torch.Size([24]).
While copying the parameter named "RCNN_rpn.RPN_cls_score.weight", whose dimensions in the model are torch.Size([18, 512, 1, 1]) and whose dimensions in the checkpoint are torch.Size([24, 512, 1, 1]).
While copying the parameter named "RCNN_rpn.RPN_bbox_pred.bias", whose dimensions in the model are torch.Size([36]) and whose dimensions in the checkpoint are torch.Size([48]).
While copying the parameter named "RCNN_rpn.RPN_bbox_pred.weight", whose dimensions in the model are torch.Size([36, 512, 1, 1]) and whose dimensions in the checkpoint are torch.Size([48, 512, 1, 1]).
While copying the parameter named "RCNN_cls_score.bias", whose dimensions in the model are torch.Size([21]) and whose dimensions in the checkpoint are torch.Size([11]).
While copying the parameter named "RCNN_cls_score.weight", whose dimensions in the model are torch.Size([21, 4096]) and whose dimensions in the checkpoint are torch.Size([11, 4096]).
While copying the parameter named "RCNN_bbox_pred.bias", whose dimensions in the model are torch.Size([84]) and whose dimensions in the checkpoint are torch.Size([44]).
While copying the parameter named "RCNN_bbox_pred.weight", whose dimensions in the model are torch.Size([84, 4096]) and whose dimensions in the checkpoint are torch.Size([44, 4096]).

[CyanideCentral]

It appears that two problems led to my error:
The different dimensions of "RCNN_rpn.RPN_cls_score.bias" (18, 24) are caused by the default ANCHOR_SCALES parameter used in demo.py. Model trained on COCO dataset has anchor_scales [4,8,16,32], while demo uses the same [8,16,32] for all datasets as defined here. Thus, to run demo on model trained on COCO, change that line to [4,8,16,32].
The different dimensions of "RCNN_rpn.RPN_cls_score.bias" (21, 11) can be fixed by changing this line to the classes in your own dataset.

[babyjie57]

@CyanideCentral thanks~

[akira-l]

@CyanideCentral got the same error, thanks!

[alicedingyueming]

may I ask you a question？How do you make vg dataset？I follow the process of bottom-up-attention， but can not get dataset as the project asked