Opencv_contrib: Opencv_dnn >> can't load network ResNet-101 for face recogniiton

[pi-null-mezon]

System information (version)

• OpenCV => 3.1
• Operating System / Platform => Windows 64 Bit
• Compiler => Visual Studio 2015

Detailed description

Hello. I have found that ResNet-101 cannot be loaded by opencv dnn module. When try to load this message appears:

[libprotobuf ERROR C:\Programming\3rdParties\opencv310\opencv_contrib\modules\dn
n\3rdparty\protobuf\sources\protobuf-3.1.0\src\google\protobuf\text_format.cc:29
8] Error parsing text-format caffe.NetParameter: 33:26: Message type "caffe.Laye
rParameter" has no field named "batch_norm_param".

OpenCV Error: Unspecified error (FAILED: ReadProtoFromTextFile(param_file, param
). Failed to parse NetParameter file: ResNet-101-deploy_augmentation.prototxt) in cv::dnn::ReadNetParamsFromTextFileOrDie, file C:\Programming\3rdParties\opencv310\opencv_contrib\mo
dules\dnn\src\caffe\caffe_io.cpp, line 1101 C:\Programming\3rdParties\opencv310\opencv_contrib\modules\dnn\src\caffe\caffe_io.cpp:1101: error: (-2) FAILED: ReadProtoFromTextFile(param_file, param). Failed to parse NetParameter file: ResNet-101-deploy_augmentation.prototxt in function cv::dnn::ReadNetParamsFromTextFileOrDie

Steps to reproduce

1) Download ResNet-101

2) Change prototxt and caffemodel file names into this example

3) Build and execute

[pi-null-mezon]

Mining deeper into the issue I have found that ResNet-101 contains "BatchNorm" and "Scale" layers that is not implemented yet in opencv_dnn module. Unfortunatelly my level of knowledges is not sufficient to make such imlementation.

[UnaNancyOwen]

The caffe parser of opencv_dnn module based on caffe old version. (Probably, It is rc2 or earlier version.)
Therefore, batch_norm_param is not supported.
I think that should update caffe.proto (and caffe.pb.h/caffe.pb.cc generated from caffe.proto).
Then, You should be update the caffe parser accordingly.
But, I'm not familiar about it. I will wait for someone contribution.
Thanks,

[arrybn]

I made all necessary to enable support of ResNet-101. But authors of the model use randomly initialized FullyConnected layer at the end (called "fc_ft"). In .caffemodel file there is no weights for this layer. In Caffe framework it works in next way: Caffe loads the model, sees that "fc_ft" doesn't have trained weights and initializes it with random. OpenCV DNN doesn't provide such a feature. So, if you download my branch with the PR and try to launch - it fails due to absence of weights for fc_ft. But still you have possibility to get inference for this model in OpenCV DNN. .caffemodel file contains weights for layer "fc_ft_augm2". You should replace all "fc_ft" by "fc_ft_augm2" in .prototxt and all will be ok

[pi-null-mezon]

Sorry, but after I have done "git clone https://github.com/arrybn/opencv_contrib.git" and build it with the sources of opencv310 in MSVC2015x86 on Windows in my test I have got this when try to load resnet101:
_[libprotobuf ERROR C:\Programming\3rdParties\opencv310\resnettest\modules\dnn\3r
dparty\protobuf\src\google\protobuf\text_format.cc:245] Error parsing text-forma
t caffe.NetParameter: 35:26: Message type "caffe.LayerParameter" has no field na
med "batch_norm_param".
OpenCV Error: Unspecified error (FAILED: ReadProtoFromTextFile(param_file, param
). Failed to parse NetParameter file: C:/Programming/3rdParties/Caffe/models/Res
Net_101_released_notrandom/ResNet-101-deploy_augmentation.prototxt) in cv::dnn::
ReadNetParamsFromTextFileOrDie, file C:\Programming\3rdParties\opencv310\resnett
est\modules\dnn\src\caffe\caffe_io.cpp, line 1101
networks is empty! Abort...Press to close this window..._

[arrybn]

Did you use issue_912 branch?

[arrybn]

Branch with fixes is here: https://github.com/arrybn/opencv_contrib/tree/issue_912

[pi-null-mezon]

No, sorry, I have used master. But, after checkout to the issue_912 I can not pass through CMake step with the following error:

_CMake Error at C:/Programming/3rdParties/opencv310/resnettest/modules/text/CMakeLists.txt:22 (ocv_add_testdata): Unknown CMake command "ocv_add_testdata"

[alalek]

Main OpenCV repository must be updated to the latest master too.

[pi-null-mezon]

Thanks, it seems I can check it out only tomorrow. I will let you know.

[pi-null-mezon]

It works! Thanks a lot! Great job!