Darknet: SlimYOLOv3: Narrower, Faster and Better for Real-Time UAV Application

[keko950]

https://arxiv.org/abs/1907.11093

[AlexeyAB]

paper: https://arxiv.org/abs/1907.11093v1

source code: https://github.com/PengyiZhang/SlimYOLOv3

---

---

[LukeAI]

Would these pruned weights work with this darknet repo. ?

[AlexeyAB]

@LukeAI It gives me an error.

So I asked about this: https://github.com/PengyiZhang/SlimYOLOv3/issues/18

---

PS

I just recompile Darknet from scratch and it works.
So this repository supports pruned models from https://github.com/PengyiZhang/SlimYOLOv3

• weights: https://github.com/PengyiZhang/SlimYOLOv3#pretrained-models
• cfg: https://github.com/PengyiZhang/SlimYOLOv3/tree/master/cfg

for example:
./darknet detector test data/drone.data prune_0.5_0.5_0.7.cfg prune_0.5_0.5_0.7_final.weights

[LukeAI]

wow, this looks it's probably the best value inference-time boost since CUDNN_HALF.

[AlexeyAB]

There is just no comparison with common datasets like MS COCO and OpenImages.

[LukeAI]

I note that they include this cfg for yolov3-spp3.cfg which achieves a higher AP that yolov3-spp.cfg maybe would be good to add to this repo.s cfg?

[keko950]

I note that they include this cfg for yolov3-spp3.cfg which achieves a higher AP that yolov3-spp.cfg maybe would be good to add to this repo.s cfg?

Did you test that cfg?

[anhuipl2010]

when the author say just add updateBN() in train(),can add it in your darknet c++ project, otherwise some others need build in pytorch env.Maybe his code cannot run,when some people found some problems. @AlexeyAB

[gmayday1997]

hi, I have added bn-pruning algorithm prune.cpp.

  ./darknet prune ./cfg/yolov3.cfg ./cfg/yolov3.weights -rate 0.3

the pruned cfg/weights are saved as ./cfg/yolov3_prune.cfg / .cfg/yolov3_prune.weights

But, some bugs need to be fixed, because something goes wrong in retraining.
I will solve it in next two days.

[AlexeyAB]

@gmayday1997

• Did you get the same good accuracy after pruning?
• Can you describe the order in which commands are run to run: training, pruning, and file tuning?

What you will fix all bugs, you can do Pull Request to this repository.

[anhuipl2010]

@gmayday1997 thanks. you're handsome.

[WongKinYiu]

it can not detect objects after pruning. (both of yolov3 and yolov3-tiny)
some models get filters=0 after pruning.

[AlexeyAB]

@WongKinYiu Do you mean https://github.com/PengyiZhang/SlimYOLOv3 or https://github.com/AlexeyAB/darknet/issues/3732#issuecomment-520365823 ?

[WongKinYiu]

@AlexeyAB i mean https://github.com/AlexeyAB/darknet/issues/3732#issuecomment-520365823
i think maybe sparse training is needed for pruning the model.

[LukeAI]

Has anybody managed to get any results using https://github.com/PengyiZhang/SlimYOLOv3 ?

[WongKinYiu]

@LukeAI
the results is good using the pruned model provided by https://github.com/PengyiZhang/SlimYOLOv3

[gmayday1997]

Did you get the same good accuracy after pruning?
Can you describe the order in which commands are run to run: training, pruning, and file tuning?

@AlexeyAB I have fixed some bugs, and fine-tuning goes well now.

|yolov3|FLOPS|Map(coco_val5k @0.5)|
|---|---|---|
| before pruned | 65 | 54.65|
|pruned w/o fine-tuning|36.3|0.2|
|pruned w fine-tuning)(2k iter)|36.3|34.6|
|pruned w fine-tuning)(9k iter)|36.3|41.3|
|pruned w fine-tuning)(12k iter)|36.3|43.4|
|pruned w fine-tuning)(28k iter)|36.3|45.2|
|pruned w fine-tuning)(update later)|

|examples(before pruned)|
|||
|---|---|---|---|
|examples(pruned w fine-tuning 9k iter)||||
|examples(pruned w fine-tuning 28k iter)| |||

|yolov3-tiny|FLOPS|Map(custom data @0.5)|
|---|---|---|
| before pruned | 7.1 | 75.4|
|pruned w/o fintuning|5.|13.5|
|pruned w fintuning)(2k iter)|5|68.1|

In addition, I added some tricks proposed in Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers

[WongKinYiu]

@gmayday1997 Hello,
do u implement overall_ratio & perlayer_ratio? (-rate in ur code is overall_ratio?)
and do u have plan to implement sparse training?

[gmayday1997]

@ WongKinYiu hi, - rate is global threshold.
local threshold pruning and sparse training inplementation are in plan.

[WongKinYiu]

@gmayday1997 thanks.

[AlexeyAB]

@gmayday1997 Thanks!

Will see, will you get the same good results as https://github.com/PengyiZhang/SlimYOLOv3
with only 10% drop in accuracy 23.9% / 26.4% mAP

[isgursoy]

watching

[dexception]

+1

[15605190982]

I have a question, this channel prune method cut off a certain percentage channels in yolo model, may cause some layers'(or almost) kernels number not the power of 2. for example, orginal layer has 256 kernels, after prune, left 111 kernels, which is not a number of the power of 2. will this hurt the inference performance? should we set the kernel number of a layer to be the power of 2 to make cudnn has the best speed?

[Hwijune]

this is my test result.

origin : 96.56 mAP 36ms
prune 95 : 94.98 mAP 10ms
prune 50 : 95.66 mAP 23ms

[15605190982]

this is my test result.
origin : 96.56 mAP 36ms
prune 95 : 94.98 mAP 10ms
prune 50 : 95.66 mAP 23ms

the mAP is so high, can you provide some information of your training data? like how many pics you use to train,to test, how many classes you have?

[Hwijune]

this is my test result.
origin : 96.56 mAP 36ms
prune 95 : 94.98 mAP 10ms
prune 50 : 95.66 mAP 23ms

the mAP is so high, can you provide some information of your training data? like how many pics you use to train,to test, how many classes you have?

using 10000 pics, val 500, used a small dataset to test

[AlexeyAB]

@hwijune

Did you use https://github.com/PengyiZhang/SlimYOLOv3 or https://github.com/gmayday1997/darknet.CG/blob/master/src/prune.cpp for pruning?

[AlexeyAB]

@gmayday1997 Hi, What final results did you get?

[Hwijune]

https://github.com/PengyiZhang/SlimYOLOv3

@AlexeyAB use https://github.com/PengyiZhang/SlimYOLOv3 repository,

yolov3 ktian08-hyp branch

Another question, do you plan to support image rotation augmentation? I want to apply it to detection.

[15605190982]

On my dataset, I got poor result. My training dataset consists of two classes, half postive samples,half negative samples, about 20k pics(10k for postive, 10k for negative). The object I need to detect is very few in real world, like cancer patients among people, which means i need to focus on both accuracy and recall. the bigger cutting ratio I set, the worse result I have.I think my experiment verified this paper https://arxiv.org/abs/1810.05270. If your dataset is sample, maybe this method will work. when i cut off 50% channels in my yolov3 model, i got 20% decrease on both accuracy and recall. training loss is around at 0.5, but before prune, the number is 0.2. If anyone has the same situation, we can discuss it. Hope this will help your guys

[gmayday1997]

hi @AlexeyAB
I make some model pruning experiments on coco. Here are the results.

|yolov3|FLOPS|Map(coco_val5k @0.5)|
|---|---|---|
| before pruned | 65 | 54.65|
|pruned @prune_rate=0.3|36.3|46.7|
|pruned @prune_rate=0.3(random pruning)|36.3|48|
|pruned @prune_rate=0.3(prune top large bn-value)|36.3|45.2|
|pruned @prune_rate=0.5(random pruning)|15|43|

On base of my experiments, I found bn-pruning is really one answer, but not the only one, because I pruned random pruning or even pruned the most top largest bn values, can also achieve good performance. It is very interesting.

[AlexeyAB]

@gmayday1997 Thanks, as I see, the drop in accuracy is quite noticeable

[LukeAI]

@gmayday1997 that is substantial - is that after sparsity training? or just after pruning?

[gmayday1997]

@AlexeyAB @LukeAI emm, yes, but depends. I found small accuracy drop on my custom dataset(12w trainval,1.7w test, 2class). Training on coco dataset is really slow, and I have only one GPU, So the finetuning iterations are all less than 8w. Maybe I need finetune more iterations.
In addition, retaining small bn-scale value and pruning large ones can also achieve comparable accuracy.

[LukeAI]

This paper suggests that a trained, pruned, fine-tuned model does not perform any better than a model using the same pruned cfg, random intitial weights and then trained from scratch. This doesn't mean that there is no value in pruning - it seems like it might be an effective way to automatically discover more efficient network architectures for a particular dataset? but it does imply that maybe pruning the same model on various datasets and comparing the results - what kind of thing tends to be preserved and what tends to be pruned away? Could be a nice way to discover a more efficient general-purpose architecture?

[AlexeyAB]

@LukeAI Conclusion: just reduce the number of filters in the middle layers and train the model from scratch

[Hwijune]

I proceeded to normal training -> parsity traing(100epoch) -> pruning -> fine tuning order. written in the paper.

[gmayday1997]

@AlexeyAB some updates(training with more iterations)

| yolov3 | volume(MB) | FLOPS |Map(coco_val5k @0.5)| finetuning iters | parameters |
|:--------------------:|:----------:|:-----:|:------------------:|:-----------------:|:------------:|
| before pruned | 246 | 65 | 54.65 | 500k | 1x |
|pruned @prune_rate=0.3| 122 | 36.3 | 48.1 | 80k | 0.5x |
|pruned @prune_rate=0.5| 60.5 | 16 | 49.2 | 160k | 0.25x |
|pruned @prune_rate=0.7| 31 | 7 | in processing | | 0.125x |

• download links to pruned cfgs/weights

  pruned @prune_rate=0.3: cfg(google driver),weight(google driver) OR cfg(baidupan)(s846),weight(baidupan)(eswd)

  pruned @prune_rate=0.5: cfg(google driver),weight(google driver) OR
  cfg(baidupan)(y9gk), weight(baidupan)(5eqt)

[LukeAI]

@gmayday1997 thanks for sharing your results! Would you be able to add a column with "inference time" or FPS?
What is the "w" unit for no. of iterations?

[gmayday1997]

@LukeAI Yes, I will test those metrics tomorrow.
'w' is just a clerical error. It is an abbreviation for 10 thousands in Chinese.
I fixed it, thank you.

[gmayday1997]

Hi, @LukeAI Here are the results about FPS test.

• speed test(experiment on 1080Ti)

| yolov3 | volume(MB) | FLOPS | FPS(352x 288) | FPS(960 x 540) | FPS(1960 x 1080) |
|:--------------------:|:----------:|:-----:|:------------------:|:-----------------:|:------------------:|
| before pruned | 246 | 65 | 60 | 57 | 53 |
|pruned @prune_rate=0.3| 122 | 36.3 | 82 | 78 | 76 |
|pruned @prune_rate=0.5| 60.5 | 16 | 107 | 105 | 97 |

[varghesealex90]

@gmayday1997 , The pipeline used is

1)Train full model

2) Prune

3) Fine-Tune pruned model

i see there the drop in accuracy is minimal when looking a the gain in FPS.

Can you please report the performance (mAP) after step 2 i.e. before fine-tuning. Using the code provided by the SLIM YOLOv3 authors, I get mAP of 0% with the prune.weights before fine-tuning. Is this normal?

[gmayday1997]

@varghesealex90 year, it is normal that the pruned model get low accuracy before fine-tuning. Based on my experiment , I found that the algorithm described blow is helpful to preserve accuracy.

[varghesealex90]

@gmayday1997 thanks for the clarification. Is the above technique implemented in https://github.com/gmayday1997/darknet.CG

[gmayday1997]

@varghesealex90 yes, here are the implementations. https://github.com/gmayday1997/darknet.CG/blob/945137080809e721f42883cbd1f7f4f7718d28f6/src/prune.cpp#L568

[dexception]

Tested Yolov3-Tiny Fine Tuning after Pruning with:
0.7 : average loss is always nan
0.5: Working fine
0.3: Working fine

[Hwijune]

@gmayday1997 Is there a difference without sparsity training?

[zpmmehrdad]

@gmayday1997 Hi,

I have added the prune.cpp to this repo and I tested it on my model with -rate 0.3 but doesn't recognize anything. I'm using TinyV3.

[WongKinYiu]

u have to retrain the model after pruning.

[zpmmehrdad]

@WongKinYiu Hi,

Could you explain in more detail? and what is the command I have to use?

[WongKinYiu]

step 1. train sparse yolo 500k epochs
step 2. prune the model
step 3. train prune yolo 500k epochs
step 4. go to step 2, or stop

[zpmmehrdad]

step 1. train sparse yolo 500k epochs
step 2. prune the model
step 3. train prune yolo 500k epochs
step 4. go to step 2, or stop

Thanks, I could do it.

[zpmmehrdad]

@gmayday1997 Hi,

Thanks for sharing this code. I could add prune.cpp to this repo and I have created a model and I've tested it with OpenCV-dnn that already takes ~500ms and now takes ~220ms on my PC. I'm using v3-Tiny network.

[varghesealex90]

step 1. train sparse yolo 500k epochs
step 2. prune the model
step 3. train prune yolo 500k epochs
step 4. go to step 2, or stop

Can you explain what step 1: train sparse yolo means?

[lq0104]

This paper suggests that a trained, pruned, fine-tuned model does not perform any better than a model using the same pruned cfg, random intitial weights and then trained from scratch. This doesn't mean that there is no value in pruning - it seems like it might be an effective way to automatically discover more efficient network architectures for a particular dataset? but it does imply that maybe pruning the same model on various datasets and comparing the results - what kind of thing tends to be preserved and what tends to be pruned away? Could be a nice way to discover a more efficient general-purpose architecture?

@LukeAI Although I didn't do slim train, but I think your words about train from prune and train from scratch is meaningful. I think the slimyolov3 paper has a logic mislead, he compared the results between yolov3-spp-prune and yolov3-tiny with nearly same FLOPs, but the reason of the results yolov3-spp-prune better than yolov3-tiny maybe because spp has deeper network architecture, rather than this slim strategy. so I think the meaningful compare is train from prune and train from scratch, but the paper doesn't supply.
On the other hand, if this paper gives a nice way to discover a more efficient general-purpose architecture, but why wouldn't I use NAS? I think the NAS technology is more mature to find efficient architecture, although this need more and more compute resource.

[LukeAI]

I got pretty good results training a slimyolo from scratch - I only lost a little bit of accuracy compared to the full-fat edition. I actually don't need the FPS and increasing image resolution doesn't help accuracy much in my use-case so I don't use it but I would use it rather than tiny-yolov3 if I needed that speed. I've If you have the GPU hours to do a NAS and discover a fantastic new efficient architecture for us then please go for it :)

[MuhammadAsadJaved]

@LukeAI
the results is good using the pruned model provided by https://github.com/PengyiZhang/SlimYOLOv3

He did not provide the pruned model. Can you send me the link where is his trained pruned model? @WongKinYiu

[WongKinYiu]

@MuhammadAsadJaved

They removed all of models in 25 Oct 2019.

[MuhammadAsadJaved]

@WongKinYiu ohh I see. OK Thank you.

[MuhammadAsadJaved]

step 1. train sparse yolo 500k epochs
step 2. prune the model
step 3. train prune yolo 500k epochs
step 4. go to step 2, or stop

Can you explain what step 1: train sparse yolo means?

@WongKinYiu @zpmmehrdad @varghesealex90
Can you please explain step 1 , what is Sparse training? I am confused. i trained original YOLOV3 on custom dataset, then pruned the model should i train it again normally ? or the sparse meaning something different?

[WongKinYiu]

@MuhammadAsadJaved

Sparse training means add sparse constraint of weights when training the model.
so there are many value of weights are near to zero, prune those weights in step 2 would not decreases so much accuracy.

[MuhammadAsadJaved]

@WongKinYiu
OK got it. How to do this procedure? i am using original YOLOV3, I already trained for 100K epoch.
Is it convenient to add wechat? if so then please add me. 13263164767

[WongKinYiu]

@MuhammadAsadJaved

I do not use wechat, and the command for doing spare training is at: https://github.com/PengyiZhang/SlimYOLOv3#2-sparsity-training
detail: https://github.com/PengyiZhang/SlimYOLOv3/tree/master/yolov3

[MuhammadAsadJaved]

@WongKinYiu OK. Thank you so much.

[MuhammadAsadJaved]

@gmayday1997 , The pipeline used is

1)Train full model

2) Prune

3) Fine-Tune pruned model

i see there the drop in accuracy is minimal when looking a the gain in FPS.

Can you please report the performance (mAP) after step 2 i.e. before fine-tuning. Using the code provided by the SLIM YOLOv3 authors, I get mAP of 0% with the prune.weights before fine-tuning. Is this normal?

@varghesealex90 Hi,
You mean you did not use the sparsity training?
(https://github.com/PengyiZhang/SlimYOLOv3#2-sparsity-training)

You only use 1-Train original YOLOV3 model, 2-Prune trained YOLOV3 model 3- Fine tune Pruned YOLOV3 model?

[joel5638]

@gmayday1997 @AlexeyAB @varghesealex90 Can we skip the first step of normal training as we already have the Yolov3.weights?

[joel5638]

@WongKinYiu When I run the pthon3 sparsity with required arguments. Nothing seems to happen. I dont see any error neither do i see any result. I just gets processed. Is that the correct way?

[WongKinYiu]

i think yes.