Detectron2: What is the Detectron2 Keypoint format and how to save it in COCO format?

[hosnasattar]

The detectron2 outputs 18 keypoints! I was wondering what are the output format and how I could save it to COCO format.
I had a look into https://github.com/facebookresearch/detectron2/pull/175
and I see that the key point output of detectron2 has 0.5 shift? Could you please clarify on this?
thanks

[ppwwyyxx]

The output format is in the documentation: https://detectron2.readthedocs.io/tutorials/models.html#model-output-format

The outputs do not have 0.5 shift. It is the COCO format that presumably has a shift.

[hosnasattar]

I can not find the ids for the keypoint! should I assume that the ids are as COCO?
"name": "person", # (specific category)
"supercategory": "person", #
"id": 1, # class id
"keypoints": [
"nose",
"left_eye",
"right_eye",
"left_ear",
"right_ear",
"left_shoulder",
"right_shoulder",
"left_elbow",
"right_elbow",
"left_wrist",
"right_wrist",
"left_hip",
"right_hip",
"left_knee",
"right_knee",
"left_ankle",
"right_ankle"
]
}

[ppwwyyxx]

If you use coco dataset, then the ids have the meaning defined by the coco dataset.
The list you provided looks right.

[hosnasattar]

I am not using COCO data set! but the dependency I am working with need as input coco format keypoints id! I am using detectron2 to get the keypoints but I was wondering if the ordering of the keypoints are the same!
for example as you can see here
the output is
Instances(num_instances=1, image_height=1100, image_width=762, fields=[pred_boxes: Boxes(tensor([[ 146.1275, 24.2675, 537.5550, 1085.6172]], device='cuda:0')), scores: tensor([0.9997], device='cuda:0'), pred_classes: tensor([0], device='cuda:0'), pred_keypoints: tensor([[[3.2279e+02, 1.4944e+02, 9.6525e-01],
[3.3422e+02, 1.3040e+02, 1.2233e+00],
[3.0660e+02, 1.3135e+02, 1.1383e+00],
[3.4279e+02, 1.3611e+02, 4.8546e-01],
[2.5708e+02, 1.3231e+02, 4.7492e-01],
[3.7136e+02, 2.4272e+02, 3.1471e-01],
[1.9327e+02, 2.5320e+02, 6.8870e-01],
[4.2470e+02, 3.7313e+02, 7.8158e-01],
[1.7041e+02, 3.8931e+02, 7.4726e-01],
[3.7613e+02, 4.5880e+02, 1.1931e+00],
[1.9517e+02, 4.6071e+02, 9.9237e-01],
[3.4375e+02, 5.2924e+02, 4.0134e-01],
[2.3041e+02, 5.3305e+02, 4.4198e-01],
[3.5708e+02, 7.6055e+02, 8.8034e-01],
[2.4946e+02, 7.6912e+02, 7.9993e-01],
[4.7994e+02, 9.7663e+02, 3.4324e-01],
[2.7327e+02, 9.9090e+02, 5.8986e-01]]], device='cuda:0')])

[ppwwyyxx]

If you use a model trained on coco dataset, then the ids also have the meaning defined by the coco dataset.

[hosnasattar]

Thanks for the support and great API :-)