Mediapipe: Simple Hand Gesture Recognition

[ChuyiZhong]

Hi all
Does anyone use this framework of mediapipe to implement a simple gesture recognition? Can it be done based on the whole library provided or other data sets are required for training?
Thanks in advance.

[loompa]

It depends on your task, if the gesture is static, like one is an open hand and other is a fist, then I think it is quite doable based on 21 x, y, z coordinate, but you still need to collect some examples of this gestures)

If gestures are dynamic I guess there is much more work need to be done here

[schumy]

I'm looking into doing the same as in here:
https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html

Where they are able to detect quite some gestures. I initially thought this was part of the library but it doesn't look like. Am I wrong?

[mgyong]

@schumy The gesture is just simple heuristics and we did not release as part of open source.
@loompa is correct that dynamic gestures is significantly harder to do than static gestures

[TheJLifeX]

I have written this
### Tutorial: Simple Hand Gesture Recognition.
You can recognize ONE, TWO, TREE, FOUR, FIVE, SIX, YEAH, ROCK, SPIDERMAN and OK like on this blog : On-Device, Real-Time Hand Tracking with MediaPipe.
Thank you for this framework.

[laisuki1109]

Hey guys, I modify the TensorFlow demo app, so it can run the palm model and hand landmark model.
please go https://github.com/laisuki1109/handtracking-with-Mediapipe to explore more !
With Java
you can set custom hand pose by xy coordinate
Thank you!

[smokingmonkey]

Hello guys. I've like to wrap this into an windows app ".exe". Is this posible? and how could i change the video source to a local video file or a remote web cam? Can u please help me with this? Thanks a lot!