Darknet: What is yolo_custom_best.weights file?

[SkuzzyxD]

I'm training a custom model on Yolo v4 for object detection, I know that Yolo saves "yolo_custom.weights" file every 100 iterations and "yolo_custom_XXXX.weights" every 1000 iterations.

Other than that, I've got a new file named "yolo_custom_best.weights" file and it keeps updating like "yolo_custom.weights" file some number of iterations.

I'm training my model using darknet with following command: ./darknet detector train "data/obj.data" "cfg/yolov4_custom.cfg" yolov4.conv.137 -dont_show -map

I've used the -map flag for first time while training now.

My questions are:

1. What is the "yolo_custom_best.weights" file? and is it formed because of me using -map flag? Is it better to detect using that file than the"yolo_custom.weights" file?
2. If I want to resume training from a certain point (if my Google Colab Runtime ends), which file should I use to resume? custom or custom_best?
3. Can some one explain me actually what is mAP? and is it causing this file to generate and update?

[flowzen1337]

1.) *_best.weights -> It is the weights file with the best tradeoff for mAP and loss which is saved seperately. If during your training a other weights file got a better tradeoff between mAP and loss, the *_best.weights file will be replaced with the new generated weights file.

2.) Use the *_last.weights file for that. The *_best.weights file could be a older file which got a good tradeoff but is x thousand itterations older than your *_last.weights file

3.) mAP = Mean Averrage Precision - In short: It describes how good your model is working on your validation dataset (test.txt). For my understanding and how i use to read it, it describes how good your weight's file currently works on your validation dataset. If you weights file classifies every annotation from your validation dataset 100%, than mAP is 100%. If it only classifies the objects inside your validation dataset 50% correctly, than your mAP is 50%. (That's how i understand it, but it's for sure possible that i'm completely wrong :-) )

[SkuzzyxD]

1.) *_best.weights -> It is the weights file with the best tradeoff for mAP and loss which is saved seperately. If during your training a other weights file got a better tradeoff between mAP and loss, the *_best.weights file will be replaced with the new generated weights file.

2.) Use the *_last.weights file for that. The *_best.weights file could be a older file which got a good tradeoff but is x thousand itterations older than your *_last.weights file

3.) mAP = Mean Averrage Precision - In short: It describes how good your model is working on your validation dataset (test.txt). For my understanding and how i use to read it, it describes how good your weight's file currently works on your validation dataset. If you weights file classifies every annotation from your validation dataset 100%, than mAP is 100%. If it only classifies the objects inside your validation dataset 50% correctly, than your mAP is 50%. (That's how i understand it, but it's for sure possible that i'm completely wrong :-) )

Ah, Thanks for the guidance 👍
After 3k iterations I got a map of 54.5% on 6 Classes.

Another small doubt, if I continue the training with *_last.weights file with the -map parameter again, the *_best.weights file be updated as well?

[flowzen1337]

yes, *_best.weights file will be replaced as soon as a better weights files will be generated.

But keep in mind, when you stop training and re-start it with your last.weight's file, i guess it will start from iteration 0 and replaces your previous generated 1000_.weights, 2000_.weights, 3000_.weights. So a new 1000_.weights will be generated which would be 4000_*.weights if you hadn't stopped and restarted the training. (I believe there is some kind of switch to keep the iteration number ongoing instead of starting at 0, but i don't know it :-) )

[SkuzzyxD]

Ah, thanks for the info.

[stephanecharette]

But keep in mind, when you stop training and re-start it with your last.weight's file, i guess it will start from iteration 0 and replaces your previous generated 1000__.weights, 2000__.weights, 3000__.weights. So a new 1000__.weights will be generated which would be 4000_*.weights if you hadn't stopped and restarted the training. (I believe there is some kind of switch to keep the iteration number ongoing instead of starting at 0, but i don't know it :-) )

I think you have this backwards. It will resume where it left off. Which is why if your .cfg says 10,000 iterations and you are already at 10K, then it immediately stops and wont train any further.

To get it to restart at zero, you must pass the -clear 1 parameter. This clears the "images seen" parameter in the header of the .weights file.

For those who are curious, the header only has a few fields:

• major version
• minor version
• patch version
• images seen

First 3 fields are 32-bit unsigned ints. Last field is 64-bit unsigned int. [See save_weights_upto() in parser.c.]

[Lowell-IC]

@stephanecharette @SkuzzyxD
Excuse me, Could you do me a favor? I don't know how to understand the three parameters: major, minor, revision. Could you tell me how to understand these?

[stephanecharette]

You don't need to understand them. They're simply the darknet "weights" version numbers.

[Lowell-IC]

You don't need to understand them. They're simply the darknet "weights" version numbers.

Thanks a lot.