Edgetpu: [edgetpu_compiler] Internal compiler error. Aborting!

[KodeWorker]

Description

I tried to convert a tensorflow model to tflite model using post-training quantization.
The conversion is completed without any error.
When I use edgetpu_compiler to compile the tflite model for TPU, it shows this error message:

Internal compiler error. Aborting!

I have no clue what cause this issue.
Need help, please.

Model

frozen pb
converted tflite

System Information

• Ubuntu 18.04
• TensorFlow 2.1.0
• Edge TPU Compiler Version: 2.1.302470888

Code for reference

I used the following script to convert frozen pb model to quantized tflite

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"  # using CPU
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import tensorflow.compat.v1 as tf
from tensorflow.python.platform import gfile
from argparse import ArgumentParser
import numpy as np

def build_argparser():
    parser = ArgumentParser()
    parser.add_argument("-m", "--model", help="Path to a .pb file with a trained model.", required=True, type=str)
    parser.add_argument("-o", "--output", help="Path to a .tflite file with a converted model.", default=None, type=str)
    return parser

def representative_dataset_gen():
    num_calibration_steps = 10
    input_size = 1024
    for _ in range(num_calibration_steps):
        # Get sample input data as a numpy array in a method of your choosing.
        input = np.random.random((1,input_size,input_size,3)).astype(np.float32)
        yield [input]

if __name__ == "__main__":
    args = build_argparser().parse_args()
    GRAPH_PB_PATH = args.model
    OUTPUT_TFLITE = args.output 

    tf.enable_eager_execution()

    with tf.Session() as sess:
        print("LOAD GRAPH")
        with gfile.FastGFile(GRAPH_PB_PATH,'rb') as f:
            graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        sess.graph.as_default()
        tf.import_graph_def(graph_def, name='')

        x = sess.graph.get_tensor_by_name("inputs:0")
        y1 = sess.graph.get_tensor_by_name("detector/yolo-v3-tiny/Conv_9/Conv2D:0")
        y2 = sess.graph.get_tensor_by_name("detector/yolo-v3-tiny/Conv_12/Conv2D:0")

        # for accelerator
        converter = tf.lite.TFLiteConverter.from_session(sess, [x], [y1, y2])
        converter.optimizations = [tf.lite.Optimize.DEFAULT]
        converter.representative_dataset = representative_dataset_gen        
        converter.target_spec.supportrf_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
        converter.inference_input_type = tf.uint8
        converter.inference_output_type = tf.uint8

        tflite_buffer = converter.convert()
        tf.gfile.GFile(OUTPUT_TFLITE, "wb").write(tflite_buffer)

    print("CONVERSION COMPLETE")

[Namburger]

@KodeWorker
Unfortunately, yolo models are not officially supported by us. You can check that from the outputs of the visualized tools, there are many ops in your model that aren't on the list of supported operations

[KodeWorker]

@Namburger
Thanks for your reply. I used Netron to view my model operations. It is true that frozen *.pb model contains many unsupported ops, but the converted tflite model should be fine. Why edgetpu_compiler fails?

[Namburger]

@KodeWorker Sorry, should have gave more details, the exact op that fails is detector/yolo-v3-tiny/Conv/LeakyRelu/alpha, you can see it in netron.
LeakyRelu isn't a supported op yet, but I don't that why it fails, something also just caught my attention, the input dimension for that layer is [1x1024x1024x32] which is probably just too big to fit in the TPU's RAM. I suggest shrinking the input size and try again!

[KodeWorker]

@Namburger Thank you for the advice. I'll try the following options and keep tracking this issue.

• [x] replace activation function with supported one
• [x] shrink the input size

Update

• Change the activation function (leaky -> relu) and retrain the model solved this issue.
• Shrink the input size to [1x384x384x3] for better runtime efficiency.
• Encountered some problems similar to this post, and load_delegate works!

[KodeWorker]

@Namburger
Thanks again for your help.
It's no longer an issue for me, but you are welcome to re-open this issue again, if there would be any support for leaky_relu.

[Namburger]

@Namburger Thank you for the advice. I'll try the following options and keep tracking this issue.

• [x] replace activation function with supported one
• [x] shrink the input size

Update

• Change the activation function (leaky -> relu) and retrain the model solved this issue.
• Shrink the input size to [1x384x384x3] for better runtime efficiency.
• Encountered some problems similar to this post, and load_delegate works!

Wow this is very good to know, so yolo model can actually be converted to over and get working with our libraries, thanks for sharing!

[Namburger]

@KodeWorker Hi again -
I kind of want to investigating more into this since you got this to work. I tried and successfully get a yolov4 to be fully quantized and even passed our compiler (prob to this repo).

Now I'm trying to evaluate the v3 model, especially the tiny-v3, would you happen to have some pointers?

[KodeWorker]

@Namburger
Sorry for the late response. I have little experience in Yolov3-tiny. According to my colleagues, the use of relu activation function rather than regular leaky relu will lead to about 10% drop in mAP.

[Namburger]

@KodeWorker thanks for the info, I think I shall try prelu in that case

[Namburger]

Hey, FYI, I found this implementation: https://github.com/guichristmann/edge-tpu-tiny-yolo