Pytorch-yolov3: recall is high and precision is very small

[hust-kevin]

[MrWhiteHomeman]

I met the same problem, did you solve it ?@hust-kevin

[hajlyx]

I think that loss_conf has something wrong, I trained a model using pjreddie's darknet code, everything goes well, and I loaded this model to continue training (using this repo), the terminal showed that loss_conf is about 31 when precision and recall is about 0.7. After training for a couple of batches the loss_conf decreased to 0.8 but the precision also decreased to 0.01. When I tested the model, it came out 27200 detections! Obviously 99% of them were false detection. And the objectness confs of these detections were about 0.999.

[fengxinjie]

I think that loss_conf has something wrong, I trained a model using pjreddie's darknet code, everything goes well, and I loaded this model to continue training (using this repo), the terminal showed that loss_conf is about 31 when precision and recall is about 0.7. After training for a couple of batches the loss_conf decreased to 0.8 but the precision also decreased to 0.01. When I tested the model, it came out 27200 detections! Obviously 99% of them were false detection. And the objectness confs of these detections were about 0.999.

the loss_cls is not right, you must change pred_cls = torch.sigmoid(prediction[..., 5:]) # Cls pred.
to pred_cls = prediction[..., 5:] # Cls pred.

sigmoid is not need for nn.CrossEntropyLoss()

[M0reDr1nk]

prediction[...,5:] in code are "the prob of each cls".
but low precision is caused by high fp.
And maybe high fp is caused by wrong conf.
so I think only changing this maybe not work...

I try to train model on my data,and meet same problem:
[Epoch 10/300, Batch 37/63] [Losses: x 0.044266, y 0.030875, w 0.077922, h 0.097630, conf 0.134416, cls 0.075356, total 0.460466, recall: 0.92308, precision: 0.10146]

Of course, I fix the code
pred_cls = prediction[..., 5:] # Cls pred.

But in test.py,it get a 'right' result.
Average Precisions:

• Class '1' - AP: 0.7774741571204787
• Class '2' - AP: 0.8802360897618701
• Class '3' - AP: 0.608286984454941
• Class '4' - AP: 0.889032905002481
• Class '5' - AP: 0.4127312412363674
• Class '6' - AP: 0.7600008860535177
  mAP: 0.7212937106049426

[jw-pyo]

Hi @hajlyx , I got the same problem in training my own dataset. Is there any progress after then?

[fengxinjie]

Hi @hajlyx , I got the same problem in training my own dataset. Is there any progress after then?

I rewrite yolov3 in pytorch, and I recommend nn.BCEWithLogitsLoss as the classifier loss, it works well

[jw-pyo]

Hi @hajlyx , I got the same problem in training my own dataset. Is there any progress after then?

I rewrite yolov3 in pytorch, and I recommend nn.BCEWithLogitsLoss as the classifier loss, it works well

Thanks for your answer @fengxinjie ! But I solved this problem by adjusting the learning rate smaller than original(0.001 -> 0.0002). And, it's a bit silly mistake, I put the annotation files at wrong directory. Hope other people will not make mistake as I've done.

[fourth-archive]

I would try this repo, P and R start high and go higher. You can plot your results as well.
https://github.com/ultralytics/yolov3

from utils import utils; utils.plot_results()