Darknet: Training stopping issue

[hafizulislamhimel45]

I am training yolov3 to detect traffic lights( single class ). I've 3000 images for training and 600 images for validation. I've trained for 4600 iterations and the current loss is .7. Loss goes up and down (.7 to .6 and .6 to .7).

for 4300 weight :

for 2700 weight:

Should I continue the training ? and The highest IoU I got 63.08%. Is it good enough or it needed to be more higher ??

[AlexeyAB]

Try to set steps=4000,5000 in cfg and train up to 6000 iterations.
Then check 2700, 3500, 4000, 4500, 5000, 5500, 6000 weights and get one with the highest mAP.

---

The highest IoU I got 63.08%. Is it good enough or it needed to be more higher ??

It depends on difficult of dataset. Also mAP is more important thatn IoU.

[hafizulislamhimel45]

Should I train from the beginning ? or From 2700 weight ?

[hafizulislamhimel45]

The object I am trying to detect is traffic light. All of the images are 1280*720. The traffic light images are very small.
This is my training set stat:

Is there anything that I can do to improve mAp & IoU ?

[AlexeyAB]

Should I train from the beginning ? or From 2700 weight ?

Try to do following recommendations and train from the begining.

https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

1. increase network resolution in your .cfg-file (height=608, width=608 or any value multiple of 32) - it will increase precision

2.

for training for small objects - set layers = -1, 11 instead of https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/cfg/yolov3.cfg#L720 and set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/cfg/yolov3.cfg#L717

[Eyshika]

@hafizulislamhimel45 : how did you decide number of iterations, since my training stopped at 100 iterations only, it is not going further