Darknet: How add 4 classes to trained(exist) weights?

[RushNuts]

Hello! How i can add 4 classes to exist weights? In my case i have 4 my classes and i want add them to yolo-voc.weights. How i can do this? Thanks

I tried add my .txt&.jpg to obj folder, create test, train; copy yolo-voc.cfg and change classes to 24, filtres to 135, after command: ./darknet partial cfg/yolo-voc-24.cfg yolo-voc.weights yolo-voc.conv.23 23
unfortunately nothing succeeded

[AlexeyAB]

No. You should train from the beginning with all 24 classes in your training dataset.

Also if you want to train 24 classes, then filters=145
(24+5)*5 = 29*5 = 145

[AlexeyAB]

1. run voc_label.py you will get: 2007_test.txt, 2007_train.txt, 2007_val.txt, 2012_train.txt, 2012_val.txt
2. run Yolo_mark to get list of jpg-files in the file train_obj.txt with full paths in a such way: yolo_mark.exe D:\Darknet2\Yolo_mark\x64\Release\data\img data/train_obj.txt data/obj.names https://github.com/AlexeyAB/Yolo_mark
3. Copy this train_obj.txt to the same directory where 2007_test.txt
4. Do type train_obj.txt 2007_train.txt 2007_val.txt 2012_*.txt > train.txt
5. Train your model with this train.txt in the obj.data file and changed classes=24 filter=145 in the yolo-voc.2.0.cfg file:
   darknet.exe detector train data/obj.data yolo-voc.2.0.cfg darknet19_448.conv.23

[lesreaper]

So @AlexeyAB , are you saying you add the custom images to the VOC images path, along with adding the custom training.txt and test.txt contents into the respective VOC versions of those files, and then train with the entire VOC dataset?

[AlexeyAB]

@RushNuts

Sorry the dumb question, train_obj.txt is it a file with my image classes?

Yes.

1. Leave here only car and person: https://github.com/AlexeyAB/darknet/blob/df076653e00db69c6fb57869981f5196a8f55e70/scripts/voc_label.py#L9
   So if there are 2 classes from voc and 4 your, then filter = 55

2. Try to use this manual for COCO training: https://pjreddie.com/darknet/yolo/#train-coco
   And https://groups.google.com/forum/#!topic/darknet/jpQHwTiC4go

3. By this way you can get pre-trained weights-file with weights only from the first 13 layers which are the same in the tiny-yolo-voc.cfg and in your tiny-yolo-obj.cfg that based on tiny-yolo-voc.cfg. Last convolutional layer is not the same in voc and your cfg files, because you change filter=55

[AlexeyAB]

@lesreaper

are you saying you add the custom images to the VOC images path, along with adding the custom training.txt and test.txt contents into the respective VOC versions of those files, and then train with the entire VOC dataset?

For example, VOC dataset can be in the directory E:\VOC2007_2012\VOCdevkit
But your dataset can be in the directory D:\Darknet2\Yolo_mark\x64\Release\data\img

So result train.txt will contain full paths to each image

E:\VOC2007_2012\VOCdevkit\VOC2007\JPEGImages\000001.jpg
D:\Darknet2\Yolo_mark\x64\Release\data\img\air1.jpg

...

[AlexeyAB]

@RushNuts
Do in the command line (on Windows or Linux):
type train_obj.txt 2007_train.txt 2007_val.txt 2012_*.txt > train.txt

[AlexeyAB]

How can I create voc.names? Since the car is 6 class, people is 14 class.

Look at the voc txt labels files VOCdevkit\VOC2007\labels, there should be 0 and 1 classes id in the txt-files.

[AlexeyAB]

@RushNuts No, there are already 0 and 1 :) Just look at these files.

voc.names should be:

person
myclass_1
myclass_2
myclass_3
myclass_4

[AlexeyAB]

@RushNuts In your case I think you can just fuse all _train, _val, _test.txt files into one, and train using it: type train_obj.txt 2007_train.txt 2007_val.txt 2012_*.txt > train.txt

for each class there are two files (one for training, another for testing). in this case, train_obj.txt comes out that contains only for learning by my classes. and test_obj.txt will not exist for my classes? sorry, I'm completely confused now trying to combine VOC classes with my classes"

Not for each class. But for each dataset. PascalVOC-2007 has (test, val, train) and PascalVOC-2012 has (val, train, test):

• train - for training
• val - for checking when you should stop training
• test - for checking detection accuracy

It is required to check which model is the best. But if you want to get the best accuracy - you should train using all datasets (test, val, train).

[AlexeyAB]

@RushNuts Yes.

[TheMikeyR]

@RushNuts in src/detector.c change the number in this line
https://github.com/AlexeyAB/darknet/blob/164ba34279a4d486d6e49409de791817b62ad1ac/src/detector.c#L182
to:

if(i >= (iter_save + 1000)) {

Remember to recompile darknet by running make in root of the darknet folder

[AlexeyAB]

@RushNuts

• So did you use 2 VOC classes and 2 your classes?
• Can you show images with bad detections?
• How many iterations did you train, what is average loss and what is mAP and IoU using ./darknet detector map obj.data obj.cfg obj.weights?

[AlexeyAB]

@RushNuts

• As I see you classes aren't detected. Only VOC classes are detected.
• Check that paths to your images there are in the train.txt that you use in the obj.data
• Try to use Yolo_mark to check your dataset: https://github.com/AlexeyAB/Yolo_mark
  ./yolo_mark /your_images_folder dont_use_train.txt /data/obj.names - can you see correct bounded boxes?

[AlexeyAB]

what classes= and filters= do you use in the cfg-file?
and show you obj.data file

Basically, my classes are detected, but the VOC classes are not always there.

As I see precision for VOC classes is good, but precision for your classes is zero.

class_id = 0, name = bus, 134682 ap = 63.88 %
class_id = 1, name = car, ap = 63.93 %
class_id = 2, name = person, ap = 60.51 %
class_id = 3, name = speed, ap = 0.00 %
class_id = 4, name = myclass2, ap = 0.00 %
class_id = 5, name = myclass3, ap = 0.00 %
class_id = 6, name = myclass4, ap = 0.00 %

[AlexeyAB]

Check, is test.txt the same as train.txt?

[AlexeyAB]

@RushNuts Copy train.txt to test.txt so it will be the same.
And then do ./darknet detector map... again.

[AlexeyAB]

So, not bad results. A little bit problems only in myclass4 and myclass2. If these classes are small, then try to increase resolution width=608 heigh=608 in cfg and do ./darknet detector map again.

---

Or you can try

• to increase resolution in the cfg width=608 heigh=608
• set random=1 in cfg
• do ./darknet detector calc_anchors data/voc.data -num_of_clusters 5 -final_width 19 -final_heigh 19 -show and then set anchors= in your cfg-file from the generated anchors.txt file
• train again about 45000 iterations

[AlexeyAB]

@RushNuts

Check that in your (not VOC) dataset there aren't non-labeled objects from VOC-classes: buses, cars, persons.

Try to train from the begining with width=608 height=608 random=1 and new anchors.

About anchors you can read: https://github.com/AlexeyAB/darknet/issues/418

[AlexeyAB]

For example, you have your images with signs Speed, but these signs are on the road where can be cars.
If these cars are not labeled in your images - this is very bad for training.

[AlexeyAB]

If the neural network sees an object that is not labeled, then it will learn to never detect it.

[AlexeyAB]

Even if this object is marked in another images.

[AlexeyAB]

Yes

[lesreaper]

@AlexeyAB I'm wondering if this might be part of my issue as well, similar to @RushNuts .

I get 1.000 mAP, tests all work, but I have terrible production use.

I'm checking people coming in and out of doors. I have thousands of images of enter and exit, as well as two custom people.

It sounds like I have to label every person in these images if I'm going to train a custom dataset using the pre-trained Tiny-Yolo conv.13 weights, even if I'm not adding "person" to my list of objects.

What if I'm also trying to train the system to recognize two specific persons, for which I have 2 classes, each with 2k images. Would I add "persons from VOC", and then add each of these individuals pictures as a separate class? Subclass?

How do I keep the system from getting confused if it is a person or a specific person?

[AlexeyAB]

@RushNuts Hi,

• what mAP can you get now?
• and what width height random params do you use?
• did you recalculate anchors for your dataset?

To increase precision you can

• just increase width height to any value multiple of 32, and do detection without re-training
• increase width height to any value multiple of 32, set random=1, and training again
• use models renset_yolo or denset_yolo, change classes, filters, num, anchors, width, height, random and train:
  
  
  
  • https://github.com/AlexeyAB/darknet/blob/master/build/darknet/x64/resnet152_yolo.cfg
  
  
  • https://github.com/AlexeyAB/darknet/blob/master/build/darknet/x64/densenet201_yolo.cfg
  
  

[AlexeyAB]

@lesreaper

I get 1.000 mAP, tests all work, but I have terrible production use.

Do you get 100% mAP and terrible production? This is very strange. It can be only if your Training dataset is very different from the Test dataset.

---

It sounds like I have to label every person in these images if I'm going to train a custom dataset using the pre-trained Tiny-Yolo conv.13 weights, even if I'm not adding "person" to my list of objects.

You should label every person on every image in two cases:

• if you want to detect persons
• if you want to detect very similar objects (person coming in door, person out of door) - in this case
  
  
  
  • obj-0: person coming in door
  
  
  • obj-1: person out of the door
  
  
  • obj-2: any other person
  
  

It is independent on do you use pre-trained tiny-yolo.conv.13.weights or not.

---

What if I'm also trying to train the system to recognize two specific persons, for which I have 2 classes, each with 2k images. Would I add "persons from VOC", and then add each of these individuals pictures as a separate class? Subclass?

So use:

• obj-0: required person-1
• obj-1: required person-2
• obj-2: any other person

[AlexeyAB]

@RushNuts

mAP is:
bus = 78.655
car = 80.57%
person = 74.59%
myclass1=0%
myclass2=0%
myclass3=0%
myclass4=0%

So your training or validation dataset does not contain images of your objects (myclass1 - 4).
You should fix it.

[AlexeyAB]

Yes, i fix it.

Can you show now mAP and AP for each class?

---

-i try change 608x608, random=1 and new anchors. When i start train in some time i get CUDA error.

Yes just can increase subdivision=16 32 or 64

[AlexeyAB]

@RushNuts This is strange that mAP lower than any AP, because mAP is average value of APs.

But result of AP is very good.

So try to train 608x608 random=1 and increased subdivision.

[AlexeyAB]

@RushNuts Hi, I think no. Gopro can't be used as WebCam or as IP-cam.
But you can use SjCam 4000 as web-cam to use with Yolo. Or use your smartphone: https://github.com/AlexeyAB/darknet#for-using-network-video-camera-mjpeg-stream-with-any-android-smartphone

[AlexeyAB]

79.01 % mAP (mean AP) doesn't look like a bad result.
You can see precision of any other detection algorithms (neural and not) using limited number of images - they have mean AP less than 79% : http://host.robots.ox.ac.uk:8080/leaderboard/displaylb.php?challengeid=11&compid=3

Do you use tiny-yolo-voc.cfg or yolo-voc.2.0.cfg?
Now you can try to use yolov3.cfg on Linux: https://github.com/pjreddie/darknet
Or wait when I will port it to Windows.

Also you should collect much more images for training.

[AlexeyAB]

@RushNuts This script for any network camera that uses HTTP-mjpeg-stream or RTSP stream.

[AlexeyAB]

@RushNuts Probably between your smartphone and PC there is NAT or proxy, so your phone doesn't have own exclusive IP, so in this case PC and Smartphone should be in the same Local Network. For example connected to the same router (Eth, Wifi)

[AlexeyAB]

@RushNuts So you should find the URL that can work in VLC.
When your phone is connected via USB - it can be in different mode, and only one of them allows IP connections.

[Vaquitta]

Hi Alexey,

This issue chain is great source of information for troubleshooting training custom set.

Had a question though.
Your comment "No. You should train from the beginning with all 24 classes in your training dataset."

Does this mean every time i have a single new class that needs to appended to existing set of classes , i have to start training from begining , with all the classes ?

[AlexeyAB]

@Vaquitta Yes, if you want good accuracy.

[Vaquitta]

Dear Alexey,

Appreciate the quick response.

If trade-off on accuracy is acceptable , how do we go about this append option. All the documents hint/instruction are for full training, hence not sure about append option..

Basically i wanted to benchmark such an "Append" vs Full training ...

[AlexeyAB]

@Vaquitta

1. Or you can just train VOC-model yolo-voc.weights on the new dataset with only 4 new classes (20, 21, 22, 23 in the txt-files).

2. Or you can train VOC-model yolo-voc.weights on the dataset that contains Pascal-VOC dataset (0-19 classes) and your dataset (20-23 classes)

3. And then compare mAP with model trained from the begining using Pascal-VOC dataset (0-19 classes) and your dataset (20-23 classes): https://github.com/AlexeyAB/darknet/tree/47c7af1cea5bbdedf1184963355e6418cb8b1b4f#how-to-train-pascal-voc-data

[Vaquitta]

Thanks Alexey.

I am planning to use the Yolov3.weights and the coco dataset.
In option 2 you meant the Yolo-voc .weights(Yolov3.weights in my case) or the base pre-trained model like the darknet53.conv.74, ideally that is what would be a training from scratch isn't it ?

[AlexeyAB]

@Vaquitta

1. yolov3.weights and modified yolov3.cfg (increased classes and filters in 3 yolo-layers)
   Train only using your dataset.

2. yolov3.conv.105 and modified yolov3.cfg (increased classes and filters in 3 yolo-layers)
   File yolov3.conv.105 you can get by command:
   darknet.exe partial cfg/yolov3.cfg yolov3.weights yolov3.conv.105 105
   Train using MSCOCO + your dataset.

3. darknet53.conv.74 and modified yolov3.cfg (increased classes and filters in 3 yolo-layers)
   Train using MSCOCO + your dataset.

[Vaquitta]

Thanks Alexey for the clarification, the picture is getting much clearer

So in all the 3 cases i would assume the
classes = 83 [Assuming i have 3 additional classes and already 80 classes of MSCOCO]
Filters = 264 [(Number of classes + 3) * 5]
In all cases i woulse use

./darknet detector train cfg/cocoCustom.data cfg/yolov3Custom.cfg [ yolov3.weights for option 1, yolov3.conv.105 for option 2 and darknet53.conv.74 for option 3]

Option 2 , i am assuming we are getting the weights other than the final detection layer and then we are trying to train the detector , am i correct..
Option 3, its pre-trained till 74 layers hence we are re-training part of classifier and detector

[AshleyRoth]

@AlexeyAB, Hi! can I add additional several classes to already trained weights? My weights, just learned. There was a need to add several more classes
Thanks!

[SmitSheth]

@AlexeyAB I also wanted to add my own custom class in yolo's object detetction model trained on COCO dataset, but without training it on whole coco datatset again. I tried to run using command : ./darknet detector train cfg/obj.data cfg/yolo-obj.cfg yolov2.weights , but the model doesn't train. It says saving weights to backup/ folder. Why isn't the model training?

[SmitSheth]

@Vaquitta Were u able to train model using option 1 as suggested by AlexeyAB i.e ./darknet detector train cfg/cocoCustom.data cfg/yolov3Custom.cfg [ yolov3.weights for option 1