Mediapipe: hand tracking, desktop version

[omnishore1]

hi is there any desktop version, i follow some instruction for desktop, but some files is missing
ERROR: Skipping 'mediapipe/examples/desktop/hand_tracking:hand_tracking_tflite': no such package 'mediapipe/examples/desktop/hand_tracking': BUILD file not found in any of the following directories.

any solution for windows desktop
thanks

[mgyong]

@omnishore1 There is only Android & iOS versions of handtracking. We do not have any desktop [linux] example for hand tracking yet. We plan to release something shortly and will update this issue when we do.
Regarding windows, pls refer to this comment #44

[LeviWadd]

Any idea when this might be (days/weeks/months) @mgyong? It would be really cool if we could have the .pb files for hand_tracking, instead of just the .tflite

[solarjoe]

I am also very interested in hand tracking for desktop. Is there a way to get this running using the e.g. the //mediapipe/models:hand_landmark.tflite and mediapipe/models:palm_detection.tflite from the Android example?

[LeviWadd]

@solarjoe You can get the hand_landmark.tflite model running in Linux by using an interpreter. Not sure if it's possible to get it running in Windows. The palm_detection model is a little more tricky as you need to provide Custom Ops. Looking at the git history, seems the TF team made steps towards making this easy, but reverted the changes.

[solarjoe]

Thanks! And I found https://github.com/wolterlw/hand_tracking and https://github.com/google/mediapipe/issues/62

[solarjoe]

@LeviWadd, what you you mean with interpreter? Tensorflow? Can you point me to some sample code?

[LeviWadd]

@solarjoe Thanks for dropping the link to that repo here. That looks very useful.

Sure, I mean something like:
(EDIT: interpreter has moved to tf.lite.Interpreter now I believe)

import numpy as np
import tensorflow as tf

# Load TFLite model and allocate tensors.
interpreter = tf.contrib.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

Taken from Nupur Garg's answer on https://stackoverflow.com/questions/50902067/how-to-import-the-tensorflow-lite-interpreter-in-python

It's also worth mentioning that if you play around with the indices of the input/output details you can get the probability that a hand is present in the supplied image.

[mgyong]

@omnishore1 @solarjoe We have just released v0.6.2 that has hand tracking desktop example. Pls check it out!