Tf-pose-estimation: Convert models into TensorFlow Lite models (.tflite)

[victor-kotov-arcadia]

Hi, could you please convert your models into TensorFlow Lite model (.tflite) format?
Or provide additional files what required for such conversion (according to the https://www.tensorflow.org/lite/devguide)

[ildoonet]

Thanks for the suggestion. I will try with it.

[haikuoyao]

I'm trying to convert it as well.
It would be great if you @ildoonet can upload your convert script and model when you finish it
And seems that there're some open model for mobile, for instance,
https://github.com/edvardHua/PoseEstimationForMobile

But it would be nice if we convert yours.
I'll share mine when I make it.

Thanks a lot.

[VincentChong123]

Hi @haikuoyao , @victor-kotov-arcadia , @ildoonet ,

I converted as follow
CMU <- success, to check accuracy
tflite_convert \
--output_file=/N/share/build/tf-pose-estimation/models/graph/cmu/cmu1.tflite \
--graph_def_file=/N/share/build/tf-pose-estimation/models/graph/cmu/graph_opt.pb \
--input_arrays=image \
--output_arrays=Openpose/concat_stage7 \
--input_shape=1,426,640,3

export dir1="/N/share/build/tf-pose-estimation/models/graph/mobilenet_thin"
tflite_convert \
--output_file=$dir1/graph.tflite \
--graph_def_file=$dir1/graph_opt.pb \
--input_arrays=image \
--input_shape=1,368,432,3 \
--output_arrays=Openpose/concat_stage7

attached below are output tflite mode and its qualitative verification code
mobilenet_thin.zip
run_tflite_pose.zip

---

mobilenet_v2_small <- error
tflite_convert \
--output_file=/N/share/build/tf-pose-estimation/models/graph/mobilenet_v2_small/graph.tflite \
--graph_def_file=/N/share/build/tf-pose-estimation/models/graph/mobilenet_v2_small/graph_opt.pb \
--input_arrays=image \
--output_arrays=Openpose/concat_stage7 \
--input_shape=1,368,432,3

error message
toco/graph_transformations/propagate_fixed_sizes.cc:118] Check failed: dim_x == dim_y (16 vs. 8)Dimensions must match

---

memo
TF version 1.14.1-dev20190414
"Openpose/concat_stage7" can be viewed from https://github.com/lutzroeder/netron node properties

ValueError: Invalid tensors 'TfPoseEstimator/Openpose/concat_stage7' were found.
FYI, I extract input/output array name from estimator.py lines below
self.tensor_image = self.graph.get_tensor_by_name('TfPoseEstimator/image:0')
self.tensor_output = self.graph.get_tensor_by_name('TfPoseEstimator/Openpose/concat_stage7:0')

Thank you

[lmy931]

@weishengchong Hi, have you tried to implement this tflite(mobilenet_thin.zip) on your mobile device?
I found the output size is 1x46x54x57. What does the 57 mean here?

[VincentChong123]

@lmy931 ,
I did not implement it on mobile device.

What does the 57 mean here?

maybe below infor is related.
https://www.learnopencv.com/multi-person-pose-estimation-in-opencv-using-openpose/
->For the 18 point model, the first 19 matrices of the output correspond to the Confidence Maps.
->The 20th to 57th matrices are the PAF matrices.

[Arth1206]

@weishengchong

Hi, thanks a lot for your help. I have a query. Can you tell me what files are "tensor_image.npy" and "tensor_output.npy" in the code "run_tflite_pose.py"?

Hoping to hear from you soon.

Thank you,
Arth

[VincentChong123]

@Arth1206,

These are resized image files as shown in code and actual file attached. Let me know if you need more infor. :)

tensor_image.zip
tensor_output.zip
estimator_bug.zip

[Arth1206]

@weishengchong

Thank you so much for that. Can you assist me a bit further?

I have converted "graph_opt.pb" file of mobilenet_thin model into TensorFlow Lite file "graph.tflite". Now, I would like to use the TFLite file to run the model for Realtime Webcam / Image inference. I tried running the "run.py" file for image inference, however I get the following error:

**Traceback (most recent call last):
File "run.py", line 37, in
e = TfPoseEstimator(get_graph_path(args.model), target_size=(432, 368))
File "/Users/arth/Desktop/tf-pose-estimation_tflite/tf_pose/estimator.py", line 324, in __init__
self.tensor_image = self.graph.get_tensor_by_name('TfPoseEstimator/image:0')
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3666, in get_tensor_by_name
return self.as_graph_element(name, allow_tensor=True, allow_operation=False)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3490, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3532, in _as_graph_element_locked
"graph." % (repr(name), repr(op_name)))

KeyError: "The name 'TfPoseEstimator/image:0' refers to a Tensor which does not exist. The operation, 'TfPoseEstimator/image', does not exist in the graph."**

I get the same error when I run “run_webcam.py” file as well. Can you please help me with that?

Thanking you in anticipation,
Arth

[VincentChong123]

Hi @Arth1206 ,

KeyError: "The name 'TfPoseEstimator/image:0' refers to a Tensor which does not exist. The operation, 'TfPoseEstimator/image', does not exist in the graph."**

check your tflite model input output nodes using https://lutzroeder.github.io/netron/
(click the input node at the top of the graph)
estimator.py should read the input tensorname as shown below
if you need more infor, please let me

[Arth1206]

@weishengchong

Thanks a lot for this. I was able to resolve the issue!

[van00]

Hi @haikuoyao , @victor-kotov-arcadia , @ildoonet ,

I converted as follow
CMU <- success, to check accuracy
tflite_convert
--output_file=/N/share/build/tf-pose-estimation/models/graph/cmu/cmu1.tflite
--graph_def_file=/N/share/build/tf-pose-estimation/models/graph/cmu/graph_opt.pb
--input_arrays=image
--output_arrays=Openpose/concat_stage7
--input_shape=1,426,640,3

export dir1="/N/share/build/tf-pose-estimation/models/graph/mobilenet_thin"
tflite_convert
--output_file=$dir1/graph.tflite
--graph_def_file=$dir1/graph_opt.pb
--input_arrays=image
--input_shape=1,368,432,3
--output_arrays=Openpose/concat_stage7

attached below are output tflite mode and its qualitative verification code
mobilenet_thin.zip
run_tflite_pose.zip

mobilenet_v2_small <- error
tflite_convert
--output_file=/N/share/build/tf-pose-estimation/models/graph/mobilenet_v2_small/graph.tflite
--graph_def_file=/N/share/build/tf-pose-estimation/models/graph/mobilenet_v2_small/graph_opt.pb
--input_arrays=image
--output_arrays=Openpose/concat_stage7
--input_shape=1,368,432,3

error message
toco/graph_transformations/propagate_fixed_sizes.cc:118] Check failed: dim_x == dim_y (16 vs. 8)Dimensions must match

memo
TF version 1.14.1-dev20190414
"Openpose/concat_stage7" can be viewed from https://github.com/lutzroeder/netron node properties

ValueError: Invalid tensors 'TfPoseEstimator/Openpose/concat_stage7' were found.
FYI, I extract input/output array name from estimator.py lines below
self.tensor_image = self.graph.get_tensor_by_name('TfPoseEstimator/image:0')
self.tensor_output = self.graph.get_tensor_by_name('TfPoseEstimator/Openpose/concat_stage7:0')

Thank you

@weishengchong hi, have you figured out how to deal with 'mobilenet_v2_small'?

[VincentChong123]

Hi @van00,

Sorry I did not implement the small version.
What error message do you receive?