Darknet: Extremely inaccurate reading.

[Soccer9001]

I been trying to train YOLO to read image of fast food trash, but YOLO constantly makes prediction in places that doesn't really make any sense whatsoever. And the more I train, the more crazier YOLO becomes.


This is with an avg loss of 0.09 and over 4000+ iteration.

I do have to mention I used 128 x 128 image to train, but this issue still pops up when I used high resolution image.

[AlexeyAB]

Check your dataset by using Yolo_mark.

[Soccer9001]

My dataset seems to be fine and mostly accurate, though is item overlapping a problem?

[AlexeyAB]

1. What mAP can you get on Training dataset?
2. What mAP can you get on Validation dataset?
3. Can you rename cfg-file to txt and attach it?
4. If you use Linux, then what parameters did you use in the Makefile?

[Soccer9001]

I am using Windows.
Can I ask how to get mAP for the dataset?
And I do not understand Question 3.

[AlexeyAB]

https://github.com/AlexeyAB/darknet#when-should-i-stop-training
darknet.exe detector map data/obj.data yolo-obj.cfg backup\yolo-obj_7000.weights

1. set valid=train.txt in obj.data file and do this command
2. set valid=valid.txt in obj.data file and do this command
3. show your cfg-file

[Soccer9001]

IoU threshold = 50 %
mean average precision ([email protected]) = 0.323183, or 32.32 %
IoU threshold = 50 %
mean average precision ([email protected]) = 0.323183, or 32.32 %

[AlexeyAB]

IoU threshold = 50 %
mean average precision ([email protected]) = 0.323183, or 32.32 %

This is not very bad accuracy for Yolo v2.

1. What command did you use for training?
2. What command do you use for detection?
3. Can you show several images with results of detection on Training images?

[Soccer9001]

1. 
   darknet.exe detector train cfg/obj.data cfg/yolo-obj.cfg yolo-obj_last.weights
   2.
   darknet.exe detector test cfg/obj.data cfg/yolo-obj.cfg yolo-obj_last.weights data/i/20.jpg
   3.
   
   
   
   

[Soccer9001]

Hello? Are you still there? I don't want to be pushy, but I really need to do this right.

[AlexeyAB]

Can you share your dataset (images, labels, train.txt, obj.data, obj.names) and your cfg-file? For example, by using Google disk?

[Soccer9001]

https://drive.google.com/drive/folders/1hA1q94x0iba1hpvbyb2z4DxbqudTWaJX?usp=sharing

[Soccer9001]

Is there anything wrong with the data I have, or do I just need to train ever longer (though I feel Yolo gets crazier when I train it for a long time)

[Soccer9001]

@AlexeyAB are you available now?

[AlexeyAB]

@Soccer9001 Hi,

Yes, your Yolo v2 model trained very bad.

Also I didn't find test.txt with Test/Valid images. And your train.txt contains much more images than you send me in the zip-archive. So i created my own mixset_train.txt

To avoid overfitting you should use 2x-4x more images, with higher resolution and use separate validation dataset to check the mAP.

---

I just trained yolov3.cfg Yolo v3 model with width=192 height=192, with recalculated anchors and pre-trained weights darknet53.conv.74 for only 1300 iterations and get mAP = 98.32% that is very high.

cfg-file, commands and mixset_train.txt: YoloStuff.zip

Detection results are pretty good:

[Soccer9001]

I am trying to re-train my data with better image quality, but I will check out the data you gave me. I will let you know if things work out.

Thank you for your help btw, I was super confused about everything and felt very useless.

[Soccer9001]

Actually, I am trying out your stuff with a 640 x 640 image now. Should I consider increasing width and height of the cfg file to something higher?

[AlexeyAB]

@Soccer9001

Yes, set batch=64 subdivisions=32 width=416 height=416 in yolov3-obj.cfg

[Soccer9001]

When I tried to run with a higher width and height, my computer ran out of memory despite setting it to subdiv 64 already, so I kept it at 192 x 192. Still, here is my result.



There are still some misread burger boxes, but overall much more stable. What should I do at this point to further improve? Should I try increasing subdiv even more so that I can increase width and height? Or do I just run Yolo for longer?

Thank you for your help as always.

[AlexeyAB]

Start training from the begining with batch=64 subdivisions=64 width=320 height=320 in yolov3-obj.cfg

And train at least 6000 iterations.

[Soccer9001]

Okay, I will do that when I get the chance.

[Soccer9001]

It is possible for me to run your yolo on a Jupyter notebook? My computer isn't the best but I have access to a good Jupyter notebook.

If so, can you teach me how to run it there? I am very new to Jupyter.

[Soccer9001]

Hey @AlexeyAB , I am trying to integrate Yolo into a html to run, so I am wondering if it is possible to run yolo detector via a python script instead of having to run it from commard prompt everytime.

[AlexeyAB]

@Soccer9001

https://github.com/AlexeyAB/darknet/blob/master/darknet.py

https://github.com/AlexeyAB/darknet/blob/master/darknet_video.py

[Soccer9001]

Hi @AlexeyAB , sorry for the long wait, I hope you are still getting this. Here is my results. I couldn't run the training program even at 320 x 320, so I went to 256 x 256 instead. The results are somewhat mixed. It is better than v2 training, but it is still making a lot of mistakes and missing a lot of objects.

[Soccer9001]

What should I do now to further improve the model?

[Soccer9001]

Here is the sample data I used to train.

https://drive.google.com/file/d/1K3Ry86WUabhH9gRsXxiyFhAf-xhmULHN/view?usp=sharing

[Soccer9001]

Hello @AlexeyAB? Are you still there?

[Soccer9001]

@AlexeyAB Hello? Maybe I should close this thread and create a new one?

[AlexeyAB]

@Soccer9001 Hi,

• Can you attach cfg-file that you used?

• Did you use random=1 for training?

• Did you use default anchors, your calculated anchors, or anchors from cfg-file from archive YoloStuff.zip https://github.com/AlexeyAB/darknet/issues/2843#issuecomment-480509467 ?

• What command did you use for training?

• What params did you use in Makefile?

[Soccer9001]

Hello @AlexeyAB

1. Here.
   ABzip.zip

2. I believe I did. But to be specific, which random are you talking about?

3. The only thing I touched from the cfg was subdiv, height and width.

4. darknet detector train obj.data yolov3-obj.cfg darknet53.conv.74 -map

5. What is a Makefile?

[AlexeyAB]

• What GPU do you use?

• Try to train with default anchors https://github.com/AlexeyAB/darknet/blob/099b71d1de6b992ce8f9d7ff585c84efd0d4bf94/cfg/yolov3.cfg#L782

[Soccer9001]

So all I have to do is replace line 782 with the line you provided me then run the training program again?

[AlexeyAB]

Do you have GTX 1050 with only 3GB GPU-RAM?

Change all 3 lines:

1. https://github.com/AlexeyAB/darknet/blob/099b71d1de6b992ce8f9d7ff585c84efd0d4bf94/cfg/yolov3.cfg#L609

2. https://github.com/AlexeyAB/darknet/blob/099b71d1de6b992ce8f9d7ff585c84efd0d4bf94/cfg/yolov3.cfg#L695

3. https://github.com/AlexeyAB/darknet/blob/099b71d1de6b992ce8f9d7ff585c84efd0d4bf94/cfg/yolov3.cfg#L782

[Soccer9001]

So only those three lines need to change, anything else?

[AlexeyAB]

Only these 3 lines

[Soccer9001]

Okay, should I continue with the 6000 int weight or start from the beginning

[AlexeyAB]

Start from the begining

[Soccer9001]

@AlexeyAB Hello again. The result is slightly better, but still have some error. Anything else I can do to improve? My mAP seems to be capped at 60% to 78%, how can I improve it?.

[AlexeyAB]

@Soccer9001 Hi,

• You can add images with slightly randomly rotated an empty tray (and empty txt label files)

• You can set:

max_batches = 12000
policy=steps
steps=10000,11000

---

• What soft did you use to get cropped objects (like cv::grabCut())?
  

• Latter I will add features: rotation and bluring for Training Yolo, so it will help too:

angles=
blur=

[Soccer9001]

I used "medibang paint pro" to crop the pictures. I will try to take pictures of an empty tray later.

Those data is meant to be in the cfg file right?

You are adding new features? Will I have to re download everything to take advantage of it? When will it be out? I have a deadline within about 2 weeks.

What should I do now? Should I restart training again or wait for your new features?

[AlexeyAB]

@Soccer9001

Those data is meant to be in the cfg file right?

Yes.

You are adding new features? Will I have to re download everything to take advantage of it? When will it be out? I have a deadline within about 2 weeks.

What should I do now? Should I restart training again or wait for your new features?

If I add this feature this week, I will tell you.

Now you can try to train by using yolov3-spp.cfg with default anchors instead of yolov3.cfg.

• Just change filters= and classes= for your number of objects in all 3 [yolo]-layers in cfg
• And set in cfg in [net] section

max_batches = 12000
policy=steps
steps=10000,11000

• Add images with empty tray
• And train from the begining with pre-trained weights http://pjreddie.com/media/files/darknet53.conv.74
  And show Loss & mAP chart.

[Soccer9001]

Is there anything I need to change in the yolov3-spp.cfg file?
Just three class and filter, then max_batch + step?
Anything about anchor needs change?

[AlexeyAB]

Is there anything I need to change in the yolov3-spp.cfg file?
Just three class and filter, then max_batch + step?

Yes

Anything about anchor needs change?

Use default

[Soccer9001]

The train process is taking a while (it is gonna take about two days to run the entire thing), but I want to make a quick update. Still has a mAp ceiling, but I haven't used the weight yet.

[Soccer9001]

Here is a link to the data set I am using.
https://drive.google.com/file/d/1xupf7i-In_PSnJjMMRqp4eiJYdNSsafq/view?usp=sharing

[Soccer9001]

Hi @AlexeyAB , bad news. I tried out the new weight, and this is overall a steo backwards. The result is similar to the one before, but now missing french fries and circling more non existent burger boxes.

[Soccer9001]

Hello @AlexeyAB, are you there? Any update on your new feature as well as what I should do from here?

[AlexeyAB]

@Soccer9001 Did you train yolov3-spp.cfg width=608 height=608 with default anchors?

[Soccer9001]

I didn't try 608x608 because I ran out of memeory with the other one. I only set to 256x256. I could try it later.

I didn't touch the anchors, do you want me to replace the anchors in the file with the one you gave me previously?

[Soccer9001]

@AlexeyAB Hello. I have tried to train with 608 x 608, unfortunately, it says Cuda is out of memory and recommend me to set subdivision to 64 even though I have already set it to 64.

[Soccer9001]

@AlexeyAB Hello? Is there anything else I can do?

[Soccer9001]

@AlexeyAB Hello? Please respond.

[AlexeyAB]

@Soccer9001 Hi,

Download the latest version of Darknet, compile it with GPU, cuDNN and OpenCV.
Take the best cfg-file that you got previously, add blur=1 parameter in the [net] section and train it from the begining.

[Soccer9001]

Hi, I am done with Yolo right now for the most part. The accuracy wasn't the best but it was acceptable at the end. If I ever need your help again, I will contact you again. Thank you for your help and I will be closing this thread now.

[AlexeyAB]

Hi, What mAP did you get at the end?