Keras-retinanet: Pretrained models for other backbone models

[ChengshuLi]

Hi,

Thank you for the great work!
Is there any chance you may release the pretrained models for other backbone models, e.g. resnet101, resnet152 or mobilenet128_1.0, mobilenet128_0.75, mobilenet160_1.0? Currently we only have pretrained models for resnet50.

That would be super helpful for transfer learning. Otherwise, I might need to train on COCO from scratch.

Thanks a lot!

[hgaiser]

Considering our resources for this project are limited, we don't provide the pretrained models for the other architectures. If in the future we happen to have trained these architectures on COCO we Will probably make them publicly available. For now, your best bet is to start with imagenet trained weights and then fine-tune on COCO or your own dataset.

[ChengshuLi]

Cool. Thanks for letting me know!

[hgaiser]

I assigned the label help wanted. We'd be happy to add pretrained (COCO/Pascal) networks to this repository if they are provided to us, but there is a risk that the architecture changes which causes those models to become obsolete. If that is the case, we likely won't update the pretrained models (except for ResNet50 on COCO).

[lvaleriu]

For training on coco what are the parameters? batch_size=1, flip_x augmentation? (if that matters)

Yes, models might change a bit. It might be a good idea to use the official keras repository models (from applications), the ones from https://github.com/keras-team/keras-contrib or copy them directly in this repository (but we still need to link to the imagenet weights).

[hgaiser]

For training on coco what are the parameters? batch_size=1, flip_x augmentation? (if that matters)

Yeah, only those.

[lvaleriu]

Started training on COCO (train2017+ val2017) using mobilenet224_1.0 + batch_size=1 + flip_x + image_min_side=800, image_max_side=1333+ perform NMS per class+ FPN correction

I'll keep updating this post with training results.

Epoch 2:
10000/10000 [==============================] - 2939s 294ms/step - loss: 3.7631 - regression_loss: 2.8676 - classification_loss: 0.8955

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.004
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.009
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.002
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.002
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.005
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.005
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.049
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.097
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.102
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.018
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.076
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.169

Epoch 4:
10000/10000 [==============================] - 2810s 281ms/step - loss: 3.3364 - regression_loss: 2.5441 - classification_loss: 0.7923

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.009
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.020
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.007
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.006
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.014
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.011
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.088
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.197
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.231
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.096
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.239
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.348

Epoch 6:
10000/10000 [==============================] - 2757s 276ms/step - loss: 3.1225 - regression_loss: 2.3957 - classification_loss: 0.7268

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.019
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.040
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.016
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.013
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.029
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.023
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.111
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.227
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.259
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.103
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.271
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.384

Epoch 8:
10000/10000 [==============================] - 4962s 496ms/step - loss: 2.9642 - regression_loss: 2.2937 - classification_loss: 0.6706

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.030
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.060
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.025
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.017
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.040
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.037
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.126
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.255
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.301
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.137
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.329
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.424

Epoch 11:
10000/10000 [==============================] - 3330s 333ms/step - loss: 2.7947 - regression_loss: 2.1787 - classification_loss: 0.6160

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.046
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.090
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.041
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.024
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.062
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.057
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.140
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.278
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.332
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.169
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.369
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.448

Epoch 13:
10000/10000 [==============================] - 28566s 3s/step - loss: 2.7168 - regression_loss: 2.1301 - classification_loss: 0.5867

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.057
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.108
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.053
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.032
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.076
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.070
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.152
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.296
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.351
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.181
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.390
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.468

Epoch 16:
10000/10000 [==============================] - 8350s 835ms/step - loss: 2.6045 - regression_loss: 2.0449 - classification_loss: 0.5596

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.067
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.128
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.063
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.034
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.084
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.084
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.160
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.300
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.347
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.186
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.381
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.455

Epoch 18:

10000/10000 [==============================] - 21270s 2s/step - loss: 2.5862 - regression_loss: 2.0303 - classification_loss: 0.5559

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.070
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.133
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.066
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.036
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.089
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.086
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.157
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.295
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.334
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.177
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.365
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.443

Epoch 20:
10000/10000 [==============================] - 2790s 279ms/step - loss: 2.4398 - regression_loss: 1.9232 - classification_loss: 0.5166

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.085
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.158
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.083
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.043
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.103
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.104
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.170
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.320
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.369
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.211
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.401
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.477

[panda9095]

@lvaleriu How is your mobilenet training process? My training on COCO using densenet169 as backbone gives only a MAP of 0.028 at epoch 24.

[lvaleriu]

@panda9095 Very bad. So i'll start again using the FPN-CORRECTION.

[hgaiser]

Actually it got merged into master.

[smehdia]

Could I use mobilennet initial weights from here?
https://github.com/experiencor/basic-yolo-keras
does it work?

[lvaleriu]

@panda9095 Started training mobilenet on coco again. I'll update the previous comment with the results after each epoch.

[lvaleriu]

@panda9095 It seems better now.
@hgaiser Can you take a look at the learning progression? I've never trained resnet50 from scratch on coco till now and dont have a reference for the learning curve.

[jjiunlin]

Here are the results after training mobilenet224_1.0 for 140+ epoch(keras-retinanet0.2,batch_size=1)
Every epoch takes 60min on my single 1080ti.The GPU utilization is 90%+.
The red line is mobilenet224_1.0 and the orange line is res50_retinanet.It seems that the loss decrease very slow.
The learning rate change because i keep training from epoch 100 using --weights command.

[sujeet-gandhi]

@lvaleriu, can u please explain why do we get 6 values of precision and recall?

[lvaleriu]

As defined in http://cocodataset.org/#detection-eval, here are the 12 metrics:

[sujeet-gandhi]

@lvaleriu Thanks.

[VCBE123]

Actually I need the more powerfull backbone support ,such as ResNeXt, or the SE-ResNeXt. Of course I tried by myself , but the performence dropped a litter while I excepted for higher. Maybe it's because that I used the customed dataset which contains about 10K images. I will train on the COCO. If there is any idea for higher performence, I would be gratefull

[hgaiser]

PRs for those backbones would be very welcome.

Pretraining on COCO sounds like the right thing to do, it also gives you a better measure of how well the backbone works.

[TimoK93]

Hey all!
I just tried to train a net with mobilenet160_0.75 as backbone. I just added "--backbone mobilenet160_0.75" to the command provided in the README.md for training on csv datasets. It is throwing an error while creating mobilenet in site-package keras-applications. Did i forget an argument?

[hgaiser]

That's better suited for a separate issue (also, mention the error, it helps to find the cause).

[scstu]

@lvaleriu could you please share with me the Pretrained models for mobilenet128_1.0 backbone ?

[liminghuiv]

For mobilenet, I saw keras-retinanet is used in vehicle detection:
https://github.com/yangliupku/retinanet_detection
Can someone merge it?

[hgaiser]

I'm closing this in favor of https://github.com/fizyr/keras-retinanet/issues/1161

[hasan-mh-aziz]

Actually I need the more powerfull backbone support ,such as ResNeXt, or the SE-ResNeXt. Of course I tried by myself , but the performence dropped a litter while I excepted for higher. Maybe it's because that I used the customed dataset which contains about 10K images. I will train on the COCO. If there is any idea for higher performence, I would be gratefull

Where did you get the pretrained weights of ResNext? Which implementation of ResNext did you follow?

[Mansi2487]

Actually I need the more powerfull backbone support ,such as ResNeXt, or the SE-ResNeXt. Of course I tried by myself , but the performence dropped a litter while I excepted for higher. Maybe it's because that I used the customed dataset which contains about 10K images. I will train on the COCO. If there is any idea for higher performence, I would be gratefull

Did you trained ResNeXt on COCO? If yes can you please provide me with the pretrained model.