Edgetpu: Internal compiler error. Aborting!

[butterflock]

I'm trying to compile a deeplab model (mobilenetv2), but I always get this error

Edge TPU Compiler version 2.0.291256449

Internal compiler error. Aborting! 

Code for the conversion to tflite. This tflite model works fine.

tflite_convert \
--graph_def_file=$INPUT \
--output_file=$OUTPUT \
  --output_format=TFLITE \
  --input_shape=1,420,513,3 \
  --input_arrays="MobilenetV2/MobilenetV2/input" \
  --inference_type=QUANTIZED_UINT8 \
  --std_dev_values=128 \
  --mean_values=128 \
  --change_concat_input_ranges=true \
  --output_arrays="ArgMax"

I'm using TF 1.15.

model.tflite

Thanks for your help!

[Namburger]

@luky152 Looks like you are hitting a previous bug by our compiler that is now fixed in our head branch. This fix will be made available with the new release, please follow our new page for updates

[butterflock]

@Namburger Thanks for your response, guess I right that the compiler git is not on github and I cannot build from sources?

[Namburger]

@luky152 unfortunately, the compiler is not opensource :/
I also suggest using post training quantization instead of the tflite_convert tool

[butterflock]

okay, initially I followed this guide in the deeplab repro (Link) abount qunatize your deeplab model.

I tried to convert the model without the tflite_convert tool, but if I convert my not quantized model the interference time is really bad (~ 4000ms). If I try to convert my quantized model trained and exported like in the link. The TFLiteConverter fails with RuntimeError: Quantization not yet supported for op: FAKE_QUANT.

Do you have an example how to convert a deeplab model to a tflite file that is compatible with the edge tpu. It should be possible since there are some edge tpu deeplab models.

[saturnin13]

+1, I critically need an answer to this. How did you guys manage to make the DeeplabV3-MobilenetV2 model work on edge TPU on your website? I have been looking for days and I cannot make it work. I have looked all over the web and I have read 20 times through all of the documentation with no luck.

[sramirez]

+1, I got the same compiler error with [EfficientNet-b1]+UNet

[lilanxiao]

+1, I got the same error even with a tiny network with just a few convolution layers and pooling. The CPU models (both float and int8 version) worked fine. l have tried tf 1.x and tf 2.x for quantization but it made no difference. (Have bought an edge-tpu, want my money back :joy: )

[znah]

Using Edge TPU Compiler version 2.1.302470888

I think this error may have something to do with activation range estimation.
Consider:

import tensorflow as tf

@tf.function
def f(x):
  return x*2.0

cf = f.get_concrete_function(tf.zeros(8))
converter = tf.lite.TFLiteConverter([cf])
def representative_dataset_gen():
  yield [tf.random.uniform([8])]  # <- Compiles successfully
  # yield [tf.zeros([8])]  # Internal compiler error. Aborting! 

converter.representative_dataset = representative_dataset_gen
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.target_spec.supported_types = [tf.int8]
tflite_quant_model = converter.convert()
open('model.tflite', 'wb').write(tflite_quant_model)

!edgetpu_compiler model.tflite

[sterlingrpi]

+4 got this error trying to compile a VGG16 SSD model. The model worked before I added the reshape and concatenate layers.

[sterlingrpi]

The edgetpu_compiler works if I take away the concatenate layer at the end and pass out all the different outputs as an array. The reshape layers seem to work just fine. With the concatenation there is no log file or information in the error message, so I am very much stuck on what to do to fix this.

Update: I got it to compile with the concatenate by reducing the number of outputs. VGG16 SSD with a 300x300 image usually has 8732. By reducing the number of filters from the first block I got 4,400 outputs which compiled successfully. Is there a hard limit I am hitting here? If so it would be very helpful for it to be specified in the documentation and error printout.

[Aspirinkb]

Same error Internal compiler error. Aborting! when I compile the deeplab_mnv2 model (quantized, QaT trained). Compiler version is 14.1.317412892

[sterlingrpi]

@Aspirinkb how many values in the output? I had an issue were it wouldn't work with too many outputs

[Aspirinkb]

@sterlingrpi output segmentation image should be 481x641, two classes. Is this size too much to compile?
I opened a new issue #172, thank you for help.

[sterlingrpi]

@Aspirinkb it shouldn't be. But for SSD I had a compilation error and reducing the number of outputs made it go away. I am also doing image segmentation and 256x256 compiles just find. Haven't tried higher. You could try reducing the image size and see if that helps. I am trying to get one of the devs to confirm if this is in fact the issue and if there is a hard limit we are running into.

[Aspirinkb]

@sterlingrpi Thank you. The input image of MobileNet v2 DeepLab v3 (1.0 depth multiplier) which in Coral official's model zoo is 513x513. So I don't think 481x641 reach EdgeTPU's limit... Anyway, I will try to reduce model's input size to see it is ok or not.

[Aspirinkb]

@sterlingrpi Yes. It works. I reduce the input size from 481x641 to 361x481 when export the model, and convert it to tflite format, and then compile successfully. I haven't study the impact of changing input size between training step and export step. Thank you again.
however, there are 6 or 7 operator not supported on edge tpu. I doubt the inference speed on edge tpu.