Darkflow: Different output from original code (darknet)

[wkentaro]

The result looks different from original code.
Anyone have idea how to fix this? I'm relatively new to darkflow.

darkflow

darknet

[hungpt297]

Same problem with yolo v2, used weight file: http://pjreddie.com/media/files/yolo.weights

[jiangsutx]

Finally, I find the problem after more than two days comparing Darknet and Darkflow.

A bug to be fixed @thtrieu

The author made a mistake in net/yolov2/test.py postprocess(), it says:
boxes = sorted(boxes, key = prob_compare)
It is designed to do NMS, but actually sorted() is to sort in ascending order!

It needs to be changed to:
boxes = sorted(boxes, key = prob_compare, reverse=True)

By doing so, the results are visually the same as Darknet version.

Other differences

• The thresh to filter bboxes is different. This value is used in test.py. And it is defined in yolo.cfg: thresh=.3.
  But in original Darknet code, it is defined as:
  float thresh = find_float_arg(argc, argv, "-thresh", .24);, by default.

I guess the thresh in yolo.cfg is only used for training. And testing can use a different one. In Darknet code, thresh for testing is not from yolo.cfg file.

• Important: imresize is very important. The Darknet 's imresize is different from others.
  I use its resized output as input to Tensorflow version, the result is the same as Darknet.
  But when I use OpenCV or scipy.misc.imresize() to resize, the results are different.
  I have no idea about the reason now.

• Tiny difference in batch_norm. Darknet implement its own batch_norm. The core part is like:
  x[index] = (x[index] - mean[f])/(sqrt(variance[f]) + .000001f);
  And Darkflow used Tensorflow's batch_norm, which is like here.
  The difference lies in the way of handle epsilon, also the value is different.

P.S.
I am translating a light version of YOLO v2 to Tensorflow.
If anyone interests, I would like to put that on github.

[thtrieu]

Wow thanks! I am fixing this ASAP.

[MisayaZ]

@jiangsutx I am interested in your light version of YOLO V2

[crazylyf]

@jiangsutx Any evaluation on the influence of all those possible reasons?

[jiangsutx]

@crazylyf The nms bug is the main problem. imresize() also influence the results. Other differences may not that important.

[crazylyf]

@jiangsutx Okay, thank you! Is there any experiment comparing the mAP on VOC or COCO dataset?

[jiangsutx]

@crazylyf No. I am not an expert in Detection. I only want to use it in my other system. : )
Please let me know, if you have any updates. Thank you.

[crazylyf]

@jiangsutx I'm not familiar with tensorflow. Instead, I'm trying to find out the discrepancy between original darknet and pytorch version of yolo2 yolo2-pytorch.