Yolov3: Detection output in coco format

[Pari-singh]

Hi @glenn-jocher, we have the option of getting our test.py results in coco (json) format. Is there a way to get the detection output as well in json format? Format like mentioned here http://cocodataset.org/#format-results

[glenn-jocher]

@Pari-singh this functionality is not available currently. Note that detect.py always runs at batch-size 1 also. The file is simply a demo of what's possible, so that you can use it as a starting point for your own development.

[joehoeller]

Ok I can modify this file, but where can I access the data needed to save the following metrics in COCO Format. (I understand I may have to do some math, but where is the raw data located for each of the following):

1. Qualitative detection output images on validation set showing true positive detections, false positives and false negatives.

2. mAP result on the validation set for IOU > 0.5.

3. Class-wise analysis of precision and recall of detections.

[glenn-jocher]

@joehoeller I don't understand your question. test.py returns all metrics on the selected dataset. See https://github.com/ultralytics/yolov3#map

[Pari-singh]

@joehoeller, data can be any on which you want to do the detection. I was working with FLIR dataset, it is available on the FLIR website.
However, getting the detection output in coco format wasn't much difficult, I took the inspiration from test.py and instead of passing the image and labels in Pytorch dataloader, I passed the image as it is and ran the model detection, saving result in json, as done in test.py (which doesn't need labels). It worked fine. As @glenn-jocher warned the detect.py runs at single image per batch, detecting it to save in coco json using test.py makes it faster.

[FranciscoReveriano]

Are you asking for a new post training grader .py file?

[joehoeller]

Ultralytics seems a lot better, I was previously using a Darknet fork that had very poor documentation. A lot of what you can do, and how to do it is (mostly) well documented here. So I think that was my main problem, was working in a forked repo.

I will look at test.py. Thank you all for your responses.

[joehoeller]

@Pari-singh excellent work btw.

[joehoeller]

@glenn-jocher Please see - https://github.com/ultralytics/yolov3/issues/650

[joehoeller]

How do we get labels for our own images? The directions are below but do not make sense to me.

Train On Custom Data

1. Convert your data to Darknet format. Your data should mirror the directory structure created by data/get_coco_dataset.sh, with images and labels in separate folders, and one label file per image.

Each image's label file must be locatable by simply replacing /images/.jpg with /labels/.txt in its pathname. An example image and label pair would be:

../coco/images/val2014/COCO_val2014_000000013992.jpg  # image
../coco/labels/val2014/COCO_val2014_000000013992.txt  # label

One file per image (if no objects in image, no label file is required).
One row per object.
Each row is class x_center y_center width height format.
Box coordinates must be in normalized xywh format (from 0 - 1).
Classes are zero-indexed (start from 0).

[joehoeller]

UPDATE: I fixed the PR and updated the math, the COCO JSON -> Darknet conversion tool (Dark Chocolate) works now: https://github.com/joehoeller/Dark-Chocolate/issues/2

[joehoeller]

We should merge Dark Chocolate into Ultralytics repo and close this out now that its vetted and tested.

[glenn-jocher]

I'll close this issue for now as the original issue appears to have been resolved, and/or no activity has been seen for some time. Feel free to comment if this is not the case.