Mmdetection: Is keypoint regressor available?

[whikwon]

Is keypoint regressor available in mmdetection?

[hellock]

Keypoint head is currently not supported, and community contribution is welcome.

[whikwon]

Thanks. I have a work keypoint head needed so that I have to implement.
If the implementation is not for the foreseeable future, I will end work and contribute.

[hellock]

That would be very nice. Thanks!

[lzhbrian]

Hi, any progress on this?

[nemonameless]

@hellock Hi, is there any plan on keypoint ?

[lxtGH]

Hi any new plan on keypoint ?

[mathmanu]

Looking forward for keypoint support!

[powerlic]

I implemented a version of mask_rcnn with keypoint and mask according to the tutorial and the detectron2, but the keypoint branch does not seem work (the loss can not converge as the detectron2), anybody could give me some advices? the link is https://github.com/powerlic/mmdetection
the start command is python train path/to/your/configs/keypoint/mask_keypoint_r50_fpn_1x_coco.py --gpus 1
@hellock

[SebastienLinker]

@powerlic Thanks for this work. I noticed you do some data augmentation during training (Resize, crops) but don't always adjust the keypoints accordingly.
The losses are producing one heatmap per keypoint, if we have 2 persons we will have 2*17 heatmaps. Shouldn't it be 17 heatmaps? (edit: nevermind, I had another paper in mind)

I changed the loss and compute the loss between 2 heatmaps instead of a heatmap and some keypoints.
I think we also need to output the keypoints.

In addition, we also have to update the drawing functions to display keypoints and the output of mmdetection inference. I suggest to output a dict with fields like keypoints, masks, bboxes ...

[SebastienLinker]

Based on @powerlic work, we redesigned the loss functions and implemented a keypoints detector based on HRNet:
https://github.com/open-mmlab/mmdetection/pull/3391