Yolov3: Cannot converge when training on my own training set.

[Ku3agi]

Thank you for your great work.
I used the yolov3-tiny network structure.
The hyper parameters are basically unchanged, except for the category setting.
I have about 700 images in 13 categories for training.
There is no bounding box output when i evaluate this model on test set,after 500 epochs training.
I do not know where is wrong here.

And this is a sample of my training set.

Need your help，please.

[github-actions[bot]]

Hello @Ku3agi, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Google Colab Notebook, Docker Image, and GCP Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

[glenn-jocher]

@Ku3agi generally if a regular person can not do the task, the model will not be able to do it either.

Since your objects are unusually shaped, you should also generate new kmeans anchors for yourself. You can use https://github.com/ultralytics/yolov3/blob/00c1fdd805a0c6bbf07d311167abbd516ff95d8b/utils/utils.py#L691-L693 for this.

[PointCloudNiphon]

when I finetuning yolov3-spp.weights with my custom dataset(a subclasses of coco(If the image contain my wanted class, it will be included)) I get this

is it normal? the precision is so low, I think..

[glenn-jocher]

@PointCloudNiphon low precision means lots of false positives.

How many classes do you have? If you have a low class count you could try training your dataset togethor as a single class using python3 train.py --single.

But It's pretty clear that the main problem is the shape of your boxes and the difficulty in your task. You need to create your own kmeans anchors, as your boxes show extreme aspect ratios compared to the pretrained coco anchors. See https://github.com/ultralytics/yolov3/issues/1009#issuecomment-609059935

You might also want to lower the iou threshold hyperparameter in train.py, perhaps to 0.1:
https://github.com/ultralytics/yolov3/blob/4fc00128293d32ae841f04ab8b6db7a32c29e153/train.py#L30

If you do all 3 of these things you will get much better results.

[glenn-jocher]

@PointCloudNiphon and of course, my last point is that you need to train to 300+ epochs. 30 will produce junk in any dataset.

If your dataset is small you should train to more epochs, i.e. 1000.

[glenn-jocher]

@PointCloudNiphon the large class loss you are seeing should be fixed now in a recent commit:
c7f93bae403ed9cf9bd50319a29485643d2438ad

Please make sure you git pull before retraining.

[PointCloudNiphon]

when I git pull your newest code,things work well, I have 7 classes.Also I will try to generate anchors as your advice...

@PointCloudNiphon the large class loss you are seeing should be fixed now in a recent commit:
c7f93ba

Please make sure you git pull before retraining.

thanks you!

[glenn-jocher]

@PointCloudNiphon hmm looks much better! I guess the class loss adjustment had a big impact.

Yes if you generate your own anchors it should also work much better.

[PointCloudNiphon]

@PointCloudNiphon hmm looks much better! I guess the class loss adjustment had a big impact.

Yes if you generate your own anchors it should also work much better.

thank you again,you are great man!

[claire-0702]

Hi , I used the yolov3-spp.cfg network structure.The hyper parameters are basically unchanged.From this picture,the precision ,f1and other values are areo.please tell me how can l solve this problem.

[glenn-jocher]

@claire-0702 see the tutorial in the wiki to get started. Look at your train and test_batch0.png files to validate your labels, and look at tensorboard or the results.png plot for your results.

From the look of things, your labels are likely incorrect. Also git pull or reclone to make sure you have the absolute latest version of the repo, things change quickly here.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove Stale label or comment or this will be closed in 5 days.