Detectron2: How do I run keypoints model using a video?

[malcolmgarner-movement]

Firstly, I am very thankful for the work done on this piece of work! It is a million times simpler to set up and run than the alternatives!

I wanted to know if I am doing the demo.py script incorrectly for videos.
I have tried running it using the --video-input tag, and the videos attempted were .mp4 .mov and .avi, but when I check the output folder, the result doesn't appear as it did for the image.

This is the command I've run:
python3.6 demo/demo.py --config-file configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x.yaml --video-input ~/IMG_2172.avi --output output/test.mp4 --opts MODEL.WEIGHTS detectron2://COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x/137261548/model_final_04e291.pkl

I believe it is a case where I'm missing something, but I'm not sure what.

Any advice or suggestions are welcomed. Let me know if any further information is needed.
Much thanks in advance.

[ppwwyyxx]

The command is correct. Could you post full logs? Have you created the directory?
Please also try a detection model to see if the issue persists.

Also, not everyone's opencv support x264 format output. In that case https://github.com/facebookresearch/detectron2/blob/3def12bdeaacd35c6f7b3b6c0097b7bc31f31ba4/demo/demo.py#L129-L131 may need to be changed.

[malcolmgarner-movement]

Hi @ppwwyyxx , thanks for the speedy response!

I have the output directory created, its in my project root for detectron2.
Log file is attached. log.txt

I tried the instance segmentation model as well, but that didn't yield any different result.

Funny that you should mention the opencv point, as this came up after initiating the run:
OpenCV: FFMPEG: tag 0x34363278/'x264' is not supported with codec id 27 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'.
I originally thought that this was a result of me using the .mp4 format, however, when I used the .avi file, this message disappeared. Also in both scenarios (where the opencv warning appeared and when it didn't) the loading bar still came up and progressed, as if it was a normal successful execution. This aspect confused me.

Let me know if any further information would be required, and thanks for your help!

[ppwwyyxx]

Just changing "x264" to some other codec (e.g. "MPEG") should work.

[malcolmgarner-movement]

Oh ok, will give it a try and update what happens

[malcolmgarner-movement]

Aha that wooorrrkkkssss!!!

Thanks a lot @ppwwyyxx!
Much appreciated. I changed it to "mp4v" and that solved it's issue with opencv for mp4