Edgetpu: What are restrictions on ResizeNearestNeighbor?

[ppershing]

Documantation states that

ResizeNearestNeighbor | All | Input/output is a 3-dimensional tensor. Depending on input/output size, this operation might not be mapped to the Edge TPU to avoid loss in precision.

I have problems with compiler crashing and my suspicion is this operation (though, it could also be MaxPool2D).
The interesting question is

• a) what combination of align_corners and half_pixel_centers does edgeTPU compiler support?
• b) are there any limitations on the dimensions? In particular, I think I might be hitting some maximum as I am upsampling a rather uneven shape of 1x4x834x128 with scale factor (1,2) to 1x4x1668x128

[Namburger]

@ppershing hi! Can you share the model?

[ppershing]

@Namburger sure. Here it is: https://www.dropbox.com/s/rpgwp9vv1s6zmak/pool.tflite?dl=0
Note that similar model, without maxpool & resize (and thus bigger intermediate tensors in "non-residual" path) compiles fine.
Also, by default tensorflow converter produces half_pixel_centers=true and the model did not work with it.
So I exchanged to false after seeing that https://github.com/guichristmann/edge-tpu-tiny-yolo used that setting successfully.

[Namburger]

@ppershing
Apologies, this is rather a bug :(
The compiler fails on this model only sometimes:

edgetpu_compiler -s pool.tflite
Edge TPU Compiler version 14.1.317412892

Internal compiler error. Aborting! 
edgetpu_compiler -s pool.tflite
Edge TPU Compiler version 14.1.317412892

Model compiled successfully in 51 ms.

Input model: pool.tflite
Input size: 594.81KiB
Output model: pool_edgetpu.tflite
Output size: 629.99KiB
On-chip memory used for caching model parameters: 78.00KiB
On-chip memory remaining for caching model parameters: 7.54MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 62
Operation log: pool_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 11
Number of operations that will run on CPU: 51

Operator                       Count      Status

RESHAPE                        1          More than one subgraph is not supported
RESHAPE                        1          Mapped to Edge TPU
ADD                            5          More than one subgraph is not supported
MAX_POOL_2D                    1          Mapped to Edge TPU
MAX_POOL_2D                    4          More than one subgraph is not supported
CONV_2D                        5          Mapped to Edge TPU
CONV_2D                        19         More than one subgraph is not supported
DEPTHWISE_CONV_2D              17         More than one subgraph is not supported
DEPTHWISE_CONV_2D              4          Mapped to Edge TPU
RESIZE_NEAREST_NEIGHBOR        5          Operation version not supported

pool.tar.gz

Also, which tensorflow version are you using for post training quantization?

[ppershing]

• Interesting - so the compiler is non-deterministic?
• I am using tf nightly 2.4.0-dev20200719 for post-trianing quantization
• RESIZE_NEAREST_NEIGHBOR 5 Operation version not supported basically says that the operation wasn't mapped. What operation "version" is then supported?

[Namburger]

@ppershing sorry, my advice for now is to stick with tensorflow2.2 until we have next compiler release, newer version runtime have special ops that our compiler hasn't caught up to yet.

[ppershing]

Just curious - any idea when we can expect next compiler release?

Anyway, I manually edited tflite & downgraded OP version from 3 to 2 + removed half_pixel_centers which seems to be unsupported in v2 of the op. Now the compiler complains about Image-interpolation layer won't run precisely enough on Edge TPU
Can you give some insides what are considered precise-enough image interpolations by the compiler?

As for the tf2.2, converter fails on converting resize neares neighbor op. I also tried with bilinear, that one works but still produces too high tflite op version for the edgetpu compiler.

[rajansaini691]

@ppershing Just curious if you were able to get ResizeNearestNeighbor to convert? Theoretically, if we could reverse-engineer the linked repo's setup, we should be able to get that performance boost. Maybe one of tensorflow's nightly builds around December 28, 2019 should do it? (I got that from the commit history)