Detectron2: How to find the keypoints for using "draw_and_connect_keypoints(keypoints)"

[AntonBaumannDE]

Hi all!
I am new to Detectron2 and am struggling a lot to visualize keypoints in my inferred images.

What I did
I inferred a model using these commands from the Colab Tutorial:

`from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
cfg = get_cfg()
cfg.merge_from_file("./detectron2_repo/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5 # set threshold for this model
cfg.MODEL.WEIGHTS = "detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl"
predictor = DefaultPredictor(cfg)
outputs = predictor(im)

outputs["instances"].pred_classes
outputs["instances"].pred_boxes

from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
v = Visualizer(im, MetadataCatalog.get("coco_2017_val"), scale=1.5)
v = v.draw_instance_predictions(outputs["instances"].to("cpu"))
v = v.draw_and_connect_keypoints(keypoints)
cv2_imshow(v.get_image()[:, :, ::-1])`

What I expected to happen / What i wish for...
I expected to draw the connected keypoints on top of my visualized InstanceSegmentation image and show it with openCV.

What happened
Console output:

`AttributeError Traceback (most recent call last)
in ()
3 v = Visualizer(im, MetadataCatalog.get("coco_2017_val"), scale=1.5)
4 v = v.draw_instance_predictions(outputs["instances"].to("cpu"))
----> 5 v = v.draw_and_connect_keypoints(keypoints)
6 cv2_imshow(v.get_image()[:, :, ::-1])

AttributeError: 'VisImage' object has no attribute 'draw_and_connect_keypoints'`

What i dont seem to get after many hours
Unfortunately I am not too familiar with Detectron. I assume that I am missing the keypoints because I am not using the right files in terms of inference and won't get the keypoints. I am running InstanceSegmentation and not KeypointSegmentation, which might be a problem. I can not figure out how to extract the keypoints and visualize them within the same framework. I also went through the documentation:
https://detectron2.readthedocs.io/modules/utils.html#module-detectron2.utils.visualizer

If anybody can help me with that, it'd be great! Tomorrow I would have a great opportunity to test it. Any advice or tip would be well appreciated!

[ppwwyyxx]

You need a different model.

cfg = get_cfg()
cfg.merge_from_file("./detectron2_repo/configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_3x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.7  # set threshold for this model
cfg.MODEL.WEIGHTS = "detectron2://COCO-Keypoints/keypoint_rcnn_R_50_FPN_3x/137849621/model_final_a6e10b.pkl"
predictor = DefaultPredictor(cfg)
outputs = predictor(im)
v = Visualizer(im[:,:,::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.5)
v = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2_imshow(v.get_image()[:, :, ::-1])

(this is now included in colab)

[AntonBaumannDE]

Great!! I couldnt find the right path for the weights, so this is perfect!! Thanks a lot! Made my day

[ppwwyyxx]

You can use any combination of config + model in https://github.com/facebookresearch/detectron2/blob/master/MODEL_ZOO.md#coco-person-keypoint-detection-baselines-with-keypoint-r-cnn