Mediapipe: How to put my own Trained model in mediapipe?

[ghost]

Hi, I'm a starter and trying to learn how to customize or modify my own mediapipe line. I used Neural Networks to train landmarks which extract from mediapipe. Is there any way I can put my trained model back to mediapipe to implementing real-time gesture recognition？Thanks for your help.

[mahakal001]

Hi @reno5566 ,
I was also trying to do the same but could not find any tutorial for that. Then I thought of first building a very simple pipeline in MediaPipe. A very simple Desktop example that accepts a video stream as input and outputs it as it is. I recently published a blog on it. In this blog, I have created the required source and BUILD files from scratch by following the MediaPipe code.
You can read it here
I am looking forward to writing one more blog about how to integrate custom tflite models.

[bhack]

We really need to lower the entry barrier for users/devs to do custom stuffs with reusable components/ops on their own model or finetuned models.

[GuodongFan]

I want to put my trained model into medipipe for dynamic gesture recognition.

[ghost]

@mahakal001 Thank you for your blog, It's really helpful. I followed your steps and get this[ERROR: bazel does not currently work properly from paths containing spaces (com.google.devtools.build.lib.runtime.BlazeWorkspace@6edf8cc7).]
Do you know how to handle this？
Looking forwward to your blog about how to integrate custom tflite models.

[mahakal001]

@reno5566 Thanks for giving it a read. As far as your issue is concerned, Can you exactly point out which line of code causes this error?
I will make sure that you get the example successfully run on your machine.

[ghost]

@mahakal001 I'm working on it but still can't find which line of code causes this error. I can understand your blog. It's strange I think it's ready to run.

[mahakal001]

@reno5566 meanwhile can you make sure that the error is not due to copy/paste formatting issues. For example, sometimes hidden characters are also copied along with the one we intended to. I am not sure but still be careful while doing so. I will attach the link to the source files in the blog itself if its a formatting issue. So please keep me updated.

[GuodongFan]

I have put a tflite neural network model on mediapipe.

My graph is configured like this：

node {
calculator: "GenVectorCalculator"
input_stream: "NORM_LANDMARKS:landmarks"
output_stream: "MATRIX:vector"
}

node {
calculator: "TfLiteConverterCalculator"
input_stream: "MATRIX:vector"
output_stream: "TENSORS:ts"
}

node {
calculator: "TfLiteInferenceCalculator"
input_stream: "TENSORS:ts"
output_stream: "TENSORS:detection_tensors"
input_side_packet: "CUSTOM_OP_RESOLVER:opresolver"
node_options: {
[type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
model_path: "mediapipe/models/action.tflite"
#use_gpu: false
}
}
}

node {
calculator: "HelloWorldCalculator"
input_stream: "TENSORS:detection_tensors"
output_stream: "RENDER_DATA:text_render_data"
}

[teafff]

Hi @reno5566 ,
I was also trying to do the same but could not find any tutorial for that. Then I thought of first building a very simple pipeline in MediaPipe. A very simple Desktop example that accepts a video stream as input and outputs it as it is. I recently published a blog on it. In this blog, I have created the required source and BUILD files from scratch by following the MediaPipe code.
You can read it here
I am looking forward to writing one more blog about how to integrate custom tflite models.

thank you for your blog.I successfully constructed your example,its very helpful.But now i want to Identify some custom objects in mediapipe.I don't know how to do it.So looking forward to your next blog.