Edgetpu: Resize Nearest Neighbor does not convert

[rajansaini691]

Hello again! I am attempting to adapt tiny yolo v3 to the edge tpu. My model quantizes and compiles, but it has inference times of ~1.5s. According to the logs, the RESIZE_NEAREST_NEIGHBOR op is not mapped to the edge device, which may account for the slowdown. What should I do to make the mapping successful and improve performance?

I am also using the newest compiler version with tensorflow 2.3.

model
compiled model
logs

[Namburger]

@rajansaini691 Thanks for reaching out!
We are actually in the experimentation stage with the yolo models also so I don't know enough to give you too much details. However there are some experimental features that I can use to allow more ops to be mapped to the tpu (not released).
Here is the compiled model with 22/25 ops mapped to the tpu it should be fast enough for real time (this is what I can offer at this time):
tiny_yolo_edgetpu.tflite.tar.gz
tiny_yolo_edgetpu.log

I'm also interested on your approach to this model, would be awesome if you could share :)

[Namburger]

@rajansaini691
FYI, the resize nearest neighbor for yolo is 4 dimensions, currently our compiler can only support 3 :(
https://coral.ai/docs/edgetpu/models-intro/#supported-operations

[rajansaini691]

@Namburger You'll find this very interesting. It turns out that the edgetpu compiler _fully_ compiles the tflite model here, which goes on to run in 27 ms! I then followed their instructions to create a retrained model for our dataset. It was a very convoluted process, but I ended up with a tflite model that was nearly identical node-for-node. The _only_ difference was that my ResizeNearestNeighbor op has half_pixel_centers set to true, while theirs is set to false. I'm not sure what this does, other than that theirs compiled while mine didn't, causing a butterfly effect and a massive performance gap.

I am not sure why this is the case. I am guessing they used an older version of the TOCO converter, but my attempts at reverse-engineering their dev environment were not very fruitful. Anyway, if either of us can figure out how to set half_pixel_centers to false, or if your team can support it being set to true, we'll be able to get tiny yolo v3 running insanely quickly and reliably.

Also, your modifications really helped. Unfortunately we are still in the process of retraining and validating our model, so I may need to send you another file in a week or so :)

[Namburger]

@rajansaini691 I see, I have also been following that same guide, but have not been able to produce the same model :(
I believe it must be related to the tensorflow version changes and how the tflite model were produced, which all comes down to experimenting at this stage.

Also, your modifications really helped. Unfortunately we are still in the process of retraining and validating our model, so I may need to send you another file in a week or so :)

Totally, keep posting the model here, I'll help you compile it!

[ZhouKai90]

@rajansaini691
FYI, the resize nearest neighbor for yolo is 4 dimensions, currently our compiler can only support 3 :(
https://coral.ai/docs/edgetpu/models-intro/#supported-operations

But Why TF2.1 can support 4-D, I am confused.

[Namburger]

@ZhouKai90 do you mean tflite converter?
Because that's a separate process from the compiler, here's the current doc:

Input/output is a 3-dimensional tensor. Depending on input/output size, this operation might not be mapped to the Edge TPU to avoid loss in precision

If you want me to compile your model with some experimental features, you can post here, I'll gladly compile it for you!

[ZhouKai90]

@ZhouKai90 do you mean tflite converter?
Because that's a separate process from the compiler, here's the current doc:

Input/output is a 3-dimensional tensor. Depending on input/output size, this operation might not be mapped to the Edge TPU to avoid loss in precision

If you want me to compile your model with some experimental features, you can post here, I'll gladly compile it for you!

I means if I install TF2.3 and use python script 'tf.lite.TFLiteConverter.from_saved_model()' to convert savedmodel to xxx.tflite. I got the "RESIZE_NEAREST_NEIGHBOR Operation version not supported" error info when i used edgetpu_compiler to compile the xxx.tflite . But if I downgraded the TF2.3 to TF2.1，the convert and compile worked perfectly without any error.

[ZhouKai90]

@ZhouKai90 do you mean tflite converter?
Because that's a separate process from the compiler, here's the current doc:
Input/output is a 3-dimensional tensor. Depending on input/output size, this operation might not be mapped to the Edge TPU to avoid loss in precision
If you want me to compile your model with some experimental features, you can post here, I'll gladly compile it for you!

I means if I install TF2.3 and use python script 'tf.lite.TFLiteConverter.from_saved_model()' to convert savedmodel to xxx.tflite. I got the "RESIZE_NEAREST_NEIGHBOR Operation version not supported" error info when i used edgetpu_compiler to compile the xxx.tflite . But if I downgraded the TF2.3 to TF2.1，the convert and compile worked perfectly without any error.

In addition, I noticed the converter between TF2.3 and TF2.1. TF2.1 is TOCO, but TF2.3 is MLIR.

[rajansaini691]

Hi @Namburger thanks for your help. Would it be alright if you could convert this model with the new compiler? Also, if it isn't too much trouble, would it be alright if you could post its inference time? I want to make sure that my environment maximizes the detection speed.

tiny_yolo_test.zip

[Namburger]

Here it is: tiny_yolo_test.tar.gz

[rajansaini691]

Hi @Namburger , would it be alright if you could compile this model as well?
tiny_yolo.zip

Also, I'm experimenting with different quantization strategies, so I may end up sending you a few (not more than 1 or 2 per day) in the near future. Is there anything I can do to make the process more streamlined for you? Or perhaps compile it myself, if the team's okay with that?

[Namburger]

tiny_yolo.tar.gz
@rajansaini691 how about send me an email?
[email protected]

[ravinemo]

@rajansaini691 Where you able to fix this issue. Our model also takes around 2 secs to inference.

[ricardodeazambuja]

Any news on this issue?

I'm using Edge TPU Compiler version 15.0.340273435 (tf.__version__ =>2.4.0) and it still gives me the message "Operation version not supported" if I try to use RESIZE_NEAREST_NEIGHBOR and RESIZE_BILINEAR.

[YsYusaito]

Hi @Namburger , @ZhouKai90
In my case, even if I used TF 2.0.0, 2.1.0 or 2.2.0, the converter was MLIR.
Apparently, default converter is MLIR.
In order to use TOCO converter, we have to write 「converter.experimental_new_converter = False」
before converting models from .h5 to .tflite(tflite_model = converter.convert()). (Please reference link below.)
https://github.com/tensorflow/tensorflow/issues/43247

I solved the RESIZE_NEAREST_NEIGHBOR problem by 「correcting gen_image_ops.py」.
･･････\Anaconda3\envs\yolov3_3\Lib\site-packages\tensorflow\python\ops\gen_image_ops.py

Correcting points are below.
half_pixel_centers=half_pixel_centers → half_pixel_centers=False
half_pixel_centers = _execute.make_bool(half_pixel_centers, "half_pixel_centers") → half_pixel_centers=False

As a side note, I eventually transformed the model under TF 2.2.0 condition and I was able to map all the inference proccessing
to Edge TPU without any problems.