Yet-another-efficientdet-pytorch: Pretrained weights for EfficientNet

[anuj-sharma-19]

Hi,

Thanks a lot for the repository. I want to initialize the model with weights from EfficientNet, rather than use the pretrained EfficientDet weights.

Could you please share the pretrained EfficientNet weights you used?

Thank You.
Anuj

[anuj-sharma-19]

Hi,

I found the pretrained weights for EfficientNet at https://github.com/lukemelas/EfficientNet-PyTorch/releases and am using the B4 pretrained weights. I have also modified your code to load it.

I am now training D4 model on coco2017 data, and the classification loss is very high (around 15-20k in value). I understand this could be possibly due to cls_loss.sum() being used in the loss rather than cls_loss.mean()?.

I actually have to train on custom dataset, without using weights trained on coco. I am hence trying to train on coco as an example before using the same procedure on my data.

It would, thus, be really great if you could guide me through how you trained your model from EfficientNet weights? Also, could you please confirm if you used the same above pretrained weights?

Thanks,
Anuj

[zylo117]

not sure what you want.
if you want to train on custom datasets, just follow the tutorial and train with pretained weights, otherwise the result will probably be bad

[anuj-sharma-19]

Thanks for your reply.

I am mainly looking to use ImageNet pretrained weights of EfficientNet to initialize the backbone of EfficientDet and train on a custom dataset. But since the loss was quite high, so thought to check the same training on coco. And found the cls_loss to be still high when using learning rate of 1e-5.

Anyways, I have increased the learning rate to 1e-3, and can see cls_loss going down quickly compared to earlier. Will see how it goes.

[edwardning]

I have similar questions.
When training my own datasets (its characteristics may be very different from coco or Imagenet), there are two ways:

1. Direct loading of efficientDET pre-trained weights for training

2. Firstly, training an efficientNET classification model on my own datasets, and then using it as the backbone to train efficientDET detection model.

Which one do you think is better? Can I share your suggestions.

[zylo117]

The first one.
Training without pretrained weights always leads to bad result, in general.

[ruodingt]

Thanks for your reply.

I am mainly looking to use ImageNet pretrained weights of EfficientNet to initialize the backbone of EfficientDet and train on a custom dataset. But since the loss was quite high, so thought to check the same training on coco. And found the cls_loss to be still high when using learning rate of 1e-5.

Anyways, I have increased the learning rate to 1e-3, and can see cls_loss going down quickly compared to earlier. Will see how it goes.

@anuj-sharma-19 Any luck mate?
I am also looking for ImageNet backbone weights.
I pretty much hit same problem as you did.

@zylo117
Thank you for the awesome work mate.

My guess is we are hitting some sort of negative transfer when using EfficientDet COCO weights.

In my previous experiment with Mask RCNN, using ImageNet weight (backbone only) rather COCO weight helps the network converge faster/better on my custom dataset. And I an also expecting to do the same in EfficientDet.

I guess you have the ImageNet weight (for the efficientNet) somewhere as it is one of the ingredients for training EfficientDet. It wold be very kind of you if you can share those weight.

Many thanks!

[zylo117]

@ruodingt https://github.com/lukemelas/EfficientNet-PyTorch/releases
you need to modify some of the weights names of the conv2staticpadding to adapt to this network.

[anuj-sharma-19]

@ruodingt I noticed the weights name mismatch in B4 yesterday, and fixed that, but am still having issues with the D4 model. The AP is still low.

[CuiZhiying]

hello, have you found out what happen? I meet with similar problem. As I mention in #347 . What really strange is when I save all Image result, and check out. It's not so bad. Many predition are reasonable.

[zylo117]

Try training in debug mode, to view the boxes. Some of them may have boxes but with wrong classes, so mAP is still low.

[anuj-sharma-19]

Hello @CuiZhiying @ruodingt it is all working good now (D4) since I fixed the weights name mismatch. I also had to change anchor sizes a bit to suit my dataset, and it all started working then.

Many thanks @zylo117 for the codebase :+1:

[Wanjpeng]

@anuj-sharma-19 Hello, I was training D4 on COCO2017 with pretrained efficientnet weights(https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b4-6ed6700e.pth). When I loading this pretrained weights there is no errors aroused in weight name mismatch. So I continued training this network. But I can't get a good result after several trials. Could you pleas tell me your training procedure or hyperparameter et.al? Thanks very much.

[anuj-sharma-19]

Hi @Wanjpeng, I did not carry forward with my training on coco, rather trained on my custom data, where I got good performance.
Maybe please check the num_classes, adv_prop setting when loading from pretrained ImageNet weights of EfficientNet (you would then see the missing_weights issue).
Also, the hyper-params in the code are same as in original paper codebase in automl; so should work on coco.

[Wanjpeng]

@anuj-sharma-19 Thanks, I will check these settings out.

[Wanjpeng]

@anuj-sharma-19 I just changed EfficientDetBackbone initializing parameters in train.py (model = EfficientDetBackbone(num_classes=len(params.obj_list), compound_coef=opt.compound_coef,
ratios=eval(params.anchors_ratios), scales=eval(params.anchors_scales),
load_weights=True)) and get _IncompatibleKeys errors. How did you do to fix these mismatch problems?

[anuj-sharma-19]

You would have to correct for the weight names in state_dict.

[Wanjpeng]

@anuj-sharma-19 Thanks, I just added a fix program to solve name mismatch problem. Here is the code (https://github.com/Wanjpeng/Fix-name-mismatch-in-loading-EfficientNet-weight-Yet-Another-EfficientDet-Pytorch-/tree/master)