Edgetpu: Internal compiler error with tf.math.multiply

[vmarkovtsev]

I have created the following model:

import tensorflow as tf

size = 1024

@tf.function(input_signature=[tf.TensorSpec([size] * 2, tf.float32)] * 2)
def bench_func(a, b):
    x = tf.math.multiply(a, b)
    return tf.reduce_sum(x)

def gen_input_samples():
    i = np.identity(size, np.float32)
    yield [-i, i]
    yield [i, i]

converter = tf.lite.TFLiteConverter.from_concrete_functions([bench_func.get_concrete_function()])
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = gen_input_samples
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()
with open("model.tflite", "wb") as fout:
    fout.write(tflite_model)

I try to convert it and the compiler crashes:

edgetpu_compiler -s model.tflite
Edge TPU Compiler version 2.0.267685300

Internal compiler error. Aborting!

File: model.zip

[Namburger]

@vmarkovtsev
From the code snippet, you are using the from_concrete_functions which is a tensorflow2.0 API. We have not started supporting post-training quantization for 2.0 yet as mentions here:

https://coral.withgoogle.com/docs/edgetpu/models-intro/#quantization

Note: To use post-training quantization, you must use TensorFlow 1.15 and set both the input and output type to uint8. (Currently, TensorFlow 2.0 does not support uint8 input/output with post-training quantization.)

[Namburger]

Here I've visualized your model using the visualize.py tool from tflite, you can see the inference input/output are still float

visualize.zip

[vmarkovtsev]

Yep I visualized myself but I thought that these quant/dequant nodes fix everything. Thanks for the hint!

OK, then I've got two questions:

• Do you think that if I manually mess with the graph it can be fixed? I'd throw away the quant/dequant and change the dtype of the inputs and the output using schema-level API.
• Is there any ETA for 2.0 support?

[Namburger]

@vmarkovtsev No problems!

Do you think that if I manually mess with the graph it can be fixed? I'd throw away the quant/dequant and change the dtype of the inputs and the output using schema-level API.

From what I can tell, manually changing the input types may actually fix it although I cannot guarantee this. Since in many cases users are using a prebuilt model where input/output cannot be changed, until post-training quantization (as provided by tf.1.15). Now your model is coming from a tf function, so I imagine this should be fine.

Is there any ETA for 2.0 support?

Sorry, no ETA yet, but I can tell you that we are working diligently to get this out. We've already fixed a lot of issues that are met from the current released compiler. Only thing I can tell you at the moment is to follow our news page, unfortunately.

[Namburger]

Sorry, super late on this, but checking this and realized the issue is with tf.reduce_sum(x)
Remove that and it will compile :)