Mediapipe: Unable to load the hand detection model

[srishtigoelroposo]

I am trying to test the given model in my sample android application.
When trying to load the model i face this issue:

java.lang.IllegalStateException: Internal error: Unexpected failure when preparing tensor allocations: Encountered unresolved custom op: Convolution2DTransposeBias.Node number 165 (Convolution2DTransposeBias) failed to prepare.

Code:
AssetFileDescriptor fileDescriptor = activity.getAssets().openFd("palm_detection.tflite");

[jiuqiant]

We have some custom ops in mediapipe/util/tflite/operations.

[srishtigoelroposo]

Is their any way i can use these custom operations and include them in my tflite setup.

i am very new to this and want help to up the model on android device.

[fanzhanggoogle]

You may find the creation of the custom op resolver here:
https://github.com/google/mediapipe/blob/731d2b95363d58f12acb29a6f8435ec33fe548d9/mediapipe/calculators/tflite/tflite_custom_op_resolver_calculator.cc#L55

And how to use it instead builtin op resolver is here:
https://github.com/google/mediapipe/blob/731d2b95363d58f12acb29a6f8435ec33fe548d9/mediapipe/calculators/tflite/tflite_inference_calculator.cc#L425

[akamzin]

Can any one give a little bit more details on how to incorporate these custom ops to be able to run it in regular python enviroment.

[Anton-Prab]

Got the same error in python when I tried to load the tflite model into the Interpreter.

ValueError: Didn't find custom op for name 'Convolution2DTransposeBias' with version 1
Registration failed. 

Code:

import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="palm_detection.tflite")

I can see that it can be resolved using the tflite::InterpreterBuilder in C api
but in python tf.lite.Interpreter alone is available InterpreterBuilder is not available

Environment:
Python - 3.7
Tensorflow - 1.14

[mlinxiang]

will is possible i compile only the tflite in mediapipe, and use this as instead of offfical tflite?

[lamarrr]

To avoid stress you can use it with Tensorflow C++ Lite. And build with Bazel.
It's faster than compiling the whole Tensorflow. I've done that and it works fine for me.
Just copy the custom op from mediapipe's tensorflow sub folder.
Then Create a class to inherit from the BuiltinOpResolver and add the Custom Op from there just as in mediapipe.

[metalwhale]

To avoid stress you can use it with Tensorflow C++ Lite. And build with Bazel.
It's faster than compiling the whole Tensorflow. I've done that and it works fine for me.
Just copy the custom op from mediapipe's tensorflow sub folder.
Then Create a class to inherit from the BuiltinOpResolver and add the Custom Op from there just as in mediapipe.

@lamarrr, and then can I load the compiled op in python by using tf.load_op_library like that?

[lamarrr]

I really don't think so because it's a Tensorflow lite package and not Tensorflow's.
And I don't think there is an API for loading ops into TensorFlow lite yet in Python, but you can do your research
cc: @fanzhanggoogle @jiuqiant

[metalwhale]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

[kulievvitaly]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

It is work for me. That is important to use python3.7 and install specified version of tensorflow

[caocao1989]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

It is work for me. That is important to use python3.7 and install specified version of tensorflow

the [repo] has been deleted,can you tell me what is the 'specified version of tensorflow' please?

[junhwanjang]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

It is work for me. That is important to use python3.7 and install specified version of tensorflow

the [repo] has been deleted,can you tell me what is the 'specified version of tensorflow' please?

@caocao1989 Tensorflow - 1.13.1 and python 3.7.3

[caocao1989]

@junhwanjang thank you very much!!!

[manchengfenxu]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

@junhwanjang , except for the 'specified version of tensorflow', is there any other modifications of the usage of tflite intenpreter? I still encountered this error：unresolved custom op: Convolution2DTransposeBias using "tensorflow-lite:1.13.0".

[scm-ns]

@jiuqiant The ops in mediapipe/util/tflite/operations seems to the cpu impls. I am wondering how the model works on the GPU?

@fanzhanggoogle I looked at the op_resolver, which seems to be used by mediapipe when it is using the GPU delegate. But the registrations for ConvTranspose is (nullptr, nullptr, nullptr, nullptr). Which means there is no function in the registration. I am curious as to where the ConvTranspose GPU kernel is implemented.

I am trying to run the TFLite model using the GPU delegate, and I am running into some errors. I am running in a EXC_BAD_ACCESS in the function Convolution2DTransposeBianParser::IsSupported. Do you guys have any info?

[shoutOutYangJie]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

@junhwanjang , except for the 'specified version of tensorflow', is there any other modifications of the usage of tflite intenpreter? I still encountered this error：unresolved custom op: Convolution2DTransposeBias using "tensorflow-lite:1.13.0".

do you solve this problem? I meet the same question.

[shoutOutYangJie]

To avoid stress you can use it with Tensorflow C++ Lite. And build with Bazel.
It's faster than compiling the whole Tensorflow. I've done that and it works fine for me.
Just copy the custom op from mediapipe's tensorflow sub folder.
Then Create a class to inherit from the BuiltinOpResolver and add the Custom Op from there just as in mediapipe.

Can tensorflow lite C++ depoly on Visual studio project and windows system?

[shoutOutYangJie]

Thanks to @junhwanjang and his awesome repo, now I can load palm detection model without custom op.

can you tell me how to solve the problem, I meet same error when running "interpreter.allocate_tensors()". The repo has been deleted, Please tell me how to avoid the problem or can you share me your tflite which can directly be used to tflite python API without C++.

[shoutOutYangJie]

@junhwanjang thank you very much!!!

can you share how to resolve this problem?

[shoutOutYangJie]

Can any one give a little bit more details on how to incorporate these custom ops to be able to run it in regular python enviroment.

have you solved this difficult question?

[shoutOutYangJie]

+

Got the same error in python when I tried to load the tflite model into the Interpreter.

ValueError: Didn't find custom op for name 'Convolution2DTransposeBias' with version 1
Registration failed. 

Code:

import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="palm_detection.tflite")

I can see that it can be resolved using the tflite::InterpreterBuilder in C api
but in python tf.lite.Interpreter alone is available InterpreterBuilder is not available

Environment:
Python - 3.7
Tensorflow - 1.14

So if I just want to run this tflite model on python environment, How to avoid this problem

[junhwanjang]

@shoutOutYangJie I closed my repo because of some private reason.
You can use "palm_detection_without_custom_op.tflite" in the below link.
https://github.com/metalwhale/hand_tracking

[shoutOutYangJie]

@shoutOutYangJie I closed my repo because of some private reason.
You can use "palm_detection_without_custom_op.tflite" in the below link.
https://github.com/metalwhale/hand_tracking

Thanks you very much. You save my life!! Could you introduce roughly what important steps you did ? How to generate this new tflite model without custom ops.

[shoutOutYangJie]

Can you share how can you avoid custom ops and run this tflite model at phthon platform?

------------------ Original ------------------
From: Junhwan Jang <[email protected]>
Date: Tue,Nov 19,2019 10:36 AM
To: google/mediapipe <[email protected]>
Cc: shoutOutYangJie <[email protected]>, Mention <[email protected]>
Subject: Re: [google/mediapipe] Unable to load the hand detection model (#35)

@shoutOutYangJie I closed my repo because of some private reason.
You can use "palm_detection_without_custom_op.tflite" in the below link.
https://github.com/metalwhale/hand_tracking

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[lip-realmax]

lo

@lamarrr Hi, could you please share a little bit more on the TF Lite C++ part? E.g. how to build and be linked with applications for C++ usage?

[Akshaysharma29]

@junhwanjang
Hi Thanks for sharing model. I have same question
https://github.com/google/mediapipe/issues/35#issuecomment-555306433

[ragavendranbala]

@Akshaysharma29 @shoutOutYangJie Did you find the solution?

[Akshaysharma29]

@ragavendranbala No I have not changed there custom_ops have you tried that?

[junhwanjang]

@Akshaysharma29 @shoutOutYangJie @ragavendranbala
Sorry for late reply first.
As I checked their operations manually, actually their custom ops (Conv2dTransposeBiasAdd) was combined with Conv2DTranspose and BiasAdd. So I "rephrased" it as Conv2DTranspose and BiasAdd separately. It was simple solution I found.

[smahadwale2001]

@Akshaysharma29 @shoutOutYangJie @ragavendranbala
Sorry for late reply first.
As I checked their operations manually, actually their custom ops (Conv2dTransposeBiasAdd) was combined with Conv2DTranspose and BiasAdd. So I "rephrased" it as Conv2DTranspose and BiasAdd separately. It was simple solution I found.

Can you share your code