Edgetpu: Inference time lower than Edge TPU performance benchmarks

[othmaneAZ]

**- Architecture: armv7l

• Running using: Tflite_runtime API
• USB 2.0**

I have mainly two questions about the EgdeTPU runtime Libraries. As I created a simple benchmark application using the Tflite_runtime C++ API to compare the performance on an embedded CPU with a USB2.0 connected EdgeTPU accelerator and the performances listed here : https://coral.ai/docs/edgetpu/benchmarks/ I'm surprised that for the mobilenetv1_244 compiled, I'm getting 9.5ms instead of 2.4ms, do you think is this huge difference normal ?

A second question, I'm a little bit worried since I'm getting the same results for both Edge TPU runtime libraries (Throttled and direct). I started thinking they are the same. Any clarifications ?

Thank you,

[Namburger]

@othmaneAZ possible duplication of https://github.com/google-coral/edgetpu/issues/115?
One big factor would be due to the USB2.0, as much data would be bottle necked by that transfer speed. What platform are you running these benchmarks on?
We found that on a pi3 it is normal to see ~9 seconds for a 224 mobilenet v1 and ~12 seconds for v2.
The benchmarks on our page were run on a robust x86 machine with usb3.0

For throttle vs direct, the throttle version only starts throttling when temperature gets too hot, at normal operating temps, I wouldn't expect much changes

[lilanxiao]

According to my experience the USB accelerator is some how sensitive to the port speed. Even with USB3 ports. Because of the extra wire length, the USB3 ports on the front panel of a PC are sometimes slower than the ports on the back panel. It is more likely to happen if the computer case is a cheap one (which means awkward wires with higher resistance). My funny observation is, the accelerator runs slower on the front panel of my PC than on the back panel. And the ports have theoretically the same technical specification!
Yeah, always make sure to use a fast port.