Darknet: recalculate anchors

[xiaosuzhang]

when i recalculate anchors for my self datasets for 416 and 416, the cmd show that:
darknet.exe detector calc_anchors data/wider_face.data -num_of_clusters 9 -width 416 -height 416
num_of_clusters = 9, width = 416, height = 13
read labels from 12880 images
loaded image: 12880 box: 159424
all loaded.

calculating k-means++ ... i = 150509, box_w = 0, box_h = 0, anchor_w = 536870912, anchor_h = -1610612736, iou = -0.511826

avg IoU = 61.35 %

so, why the value of height in the calculation process is inconsistent with the fixed value ???
Thanks!

[AlexeyAB]

so, why the value of height in the calculation process is inconsistent with the fixed value ???

1. What do you mean?

i = 150509, box_w = 0, box_h = 0, anchor_w = 536870912, anchor_h = -1610612736, iou = -0.511826

1. One of you labels is wrong.

2. Show what anchors did you get?

[xiaosuzhang]

i use the command : darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 to recalculate the anchors. But the result shows that heights=13???
why does it happen??

[AlexeyAB]

@xiaosuzhang Update your code from GitHub

[xiaosuzhang]

E:\darknet-master\build\darknet\x64>
darknet.exe detector calc_anchors data/wider_face.data -num_of_clusters 9 -width 416 -height 416

num_of_clusters = 9, width = 416, height = 13
read labels from 12880 images
loaded image: 12880 box: 159424
all loaded.

calculating k-means++ ... i = 150509, box_w = 0, box_h = 0, anchor_w = 536870912, anchor_h = -1610612736, iou = -0.511826

avg IoU = 61.35 %

Saving anchors to the file: anchors.txt
anchors = 3.7606,0.2263, 8.7632,0.5101, 16.1529,0.9067, 27.5527,1.4898, 45.1132,2.2311, 71.5754,3.3816, 110.2348,4.7562, 163.7577,6.0963, 238.9554,7.6555

[AlexeyAB]

@xiaosuzhang Update your code from GitHub

[PacteraKun]

@AlexeyAB

Here's my results.

./darknet detector calc_anchors '/home/kun/darknet/data/ped.data' -num_of_clusters 9 -width 1024 -height 572 -show

num_of_clusters = 9, width = 1024, height = 572
read labels from 11500 images
loaded image: 11499 box: 46513
all loaded.

calculating k-means++ ...
avg IoU = 73.99 %

Saving anchors to the file: anchors.txt
anchors = 18839.1270,25754.9219, 33607.5977,44379.2695, 49706.3086,65268.1055, 65878.2812,91669.6094, 85944.8828,127390.7969, 129411.4844,89209.3906, 119262.1797,169088.7969, 252382.0938,144432.7031, 159489.8906,228602.1875

Why are my anchor values so large?

[AlexeyAB]

@PacteraKun Do you use the latest code from this repo?
Looks like your labels are wrong, use: https://github.com/AlexeyAB/Yolo_mark

[PacteraKun]

@AlexeyAB
Yes, there is something inconsistent with yolo annotation format. Mine is following:
lable bottom_left_x bottom_left_y width height

Thanks.

[AlexeyAB]

@PacteraKun x,y,w,h should be relative values. And x,y - should be a center of object: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

Where:

• <object-class> - integer number of object from 0 to (classes-1)
• <x> <y> <width> <height> - float values relative to width and height of image, it can be equal from (0.0 to 1.0]
• for example: <x> = <absolute_x> / <image_width> or <height> = <absolute_height> / <image_height>
• atention: <x> <y> - are center of rectangle (are not top-left corner)

For example for img1.jpg you will be created img1.txt containing:

1 0.716797 0.395833 0.216406 0.147222 0 0.687109 0.379167 0.255469 0.158333 1 0.420312 0.395833 0.140625 0.166667

[PacteraKun]

@AlexeyAB
Thanks for reminding me relative values. It works.

[xiaosuzhang]

Thanks, it worked! And this is my result:
E:\code\darknet\build\darknet\x64>
darknet.exe detector calc_anchors data/wider_face.data -num_of_clusters 9 -width 416 -height 416

num_of_clusters = 9, width = 416, height = 416
read labels from 12880 images
loaded image: 12880 box: 159424
all loaded.

calculating k-means++ ... i = 150509, box_w = 0, box_h = 0, anchor_w = -2147483648, anchor_h = 0, iou = -0.520745

avg IoU = 61.89 %

Saving anchors to the file: anchors.txt
anchors = 3.9338,6.8027, 9.0820,16.5471, 17.1835,30.7406, 31.0513,50.5596, 48.3849,82.1778, 81.6062,124.8006, 98.9854,212.3232, 175.1285,168.5064, 194.1013,311.2147

Is the result good???

[AlexeyAB]

@xiaosuzhang
Yes, it is good result.

[RyanCV]

Hi, @xiaosuzhang, @AlexeyAB
I am also running the command as following:
./darknet detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416
But I got the errors:

 num_of_clusters = 9, width = 416, height = 416
train: Using default 'data/train.list'
Couldn't open file: data/train.list

In my file data/obj.data, the format in each row looks like:
/darknet/VOCdevkit/VOC2007/JPEGImages/000122.jpg
/darknet/VOCdevkit/VOC2007/JPEGImages/000129.jpg

Do you have any suggestions on running this to get the right anchor size? Thanks.

[RyanCV]

It works when I change the name of my file obj.data to train.list. I checked the code is hard coded.

[AlexeyAB]

@RyanCV

In my file data/obj.data, the format in each row looks like:
/darknet/VOCdevkit/VOC2007/JPEGImages/000122.jpg
/darknet/VOCdevkit/VOC2007/JPEGImages/000129.jpg

It's wrong, read carefully: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

1. Create file obj.data in the directory build\darknet\x64data\, containing (where classes = number of objects):

classes= 2
train  = data/train.txt
valid  = data/test.txt
names = data/obj.names
backup = backup/

...

1. Create file train.txt in directory build\darknet\x64data\, with filenames of your images, each filename in new line, with path relative to darknet.exe, for example containing:

data/obj/img1.jpg
data/obj/img2.jpg
data/obj/img3.jpg

[moyans]

@AlexeyAB when I use this script :darknet.exe detector calc_anchors data/wider_face.data -num_of_clusters 9 -width 416 -height 416 means the higheravg IoU is the better? In other words, when I get a low avg IoU, It means I need to add anchor number?

[AlexeyAB]

@moyans In general - yes. Then you should change anchors=, num=, masks=, filters= in 3 places in the cfg-file.

[PacteraKun]

@AlexeyAB
What happened the performance of YOLO V3 getting worse when I used the generated anchors, whose avg IoU is more than 70%, than when I used the default anchors?

[AlexeyAB]

@PacteraKun What mAP can you get before and after this?

[moyans]

@AlexeyAB Thank you for your reply. once num_of_clusters 9 the avg IoU only 60%， when i increase num_of_clusters to 18, get avg IoU 78%, I know how to change anchors=, num=, filters= , but masks= is also to change? and how to change it?

[AlexeyAB]

@moyans
Just share masks equaly, large values in the first layer.

[yolo]
masks=12, 13, 14, 15, 16, 17
...
[yolo]
masks=6, 7, 8, 9, 10, 11
...
[yolo]
masks=0, 1, 2, 3, 4, 5

And filters=(classes+5)x6

[moyans]

@AlexeyAB thanks again

[jiqiyang]

@AlexeyAB My image size is 1280x720 and my yolo3 network size is the default 416x416 ('random=1' in .cfg file).
Which command should I use to find out the proper anchor sizes for Yolo3?

./darknet detector calc_anchors data/sample.data -num_of_clusters 9 -width 1280 -height 720

or

./darknet detector calc_anchors data/sample.data -num_of_clusters 9 -width 416 -height 416

?
I guess I need -width 416 -height 416 . Is that right?

[moyans]

@jiqiyang yes, it depends on the setting of your network script, not the original image size.

[AlexeyAB]

@jiqiyang You need -width 416 -height 416 from your cfg-file:

https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

recalculate anchors for your dataset for width and height from cfg-file: darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 then set the same 9 anchors in each of 3 [yolo]-layers in your cfg-file

[jiqiyang]

Thank you, guys :) @moyans @AlexeyAB

[PacteraKun]

@AlexeyAB
After I change the config file, getting my own anchors improves the performance almost 10 map.

[daiwc]

@AlexeyAB why do i get some strange achor values different form the default value in yolov3.cfg?
The anchor was computed on VOC.
darknet.exe detector calc_anchors data/voc.data -num_of_clusters 9 -width 416 -height 416

num_of_clusters = 9, width = 416, height = 416
read labels from 16551 images
loaded image: 16551 box: 40058
all loaded.

calculating k-means++ ...
avg IoU = 66.22 %

Saving anchors to the file: anchors.txt
anchors = 40.3128,53.0350, 65.6103,123.6124, 94.6806,215.2469, 169.2410,122.0826, 134.0053,325.7523, 216.7285,228.7580, 354.5798,205.6504, 255.2334,355.9286, 378.3310,368.1178

Is the default anchors computed on COCO?
Hoping for your answer! Thanks a lot!

[AlexeyAB]

@daiwc

Is the default anchors computed on COCO?

Yes.

Also there are a littlbe bit differences in darknet.exe detector calc_anchors and gen_anchors.py/kmeansiou.c

[daiwc]

Thanks you very much!