Edgetpu: Can not allocate tensor on Windows.

[mattn]

I wrote object detector using webcam and acceralated with Coral EdgeTPU based on my code.

https://github.com/mattn/webcam-detect-tflite

Modified for edgetpu

https://gist.github.com/819be2f3c70379a659984aa199d756e0

Compilation was succeeded but when I run the app, AllocateTensor failed.

Loading model: mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite
ERROR: Internal: :159 batches * single_input_size != input->bytes (307200 != 8136)
ERROR: Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare.                                                                                            

What is wrong? I can confirm that my code which do similar thing on python works correctly. I tried master branch of tensorflow and d855adfc5a0195788bf5f92c3c7352e638aa1109 both.

EDIT: FYI, this cpp code works fine on Linux.

[mattn]

Sorry, I tried d855adfc5a0195788bf5f92c3c7352e638aa1109 again. And I got it work. Now go-tflite support Coral Edge TPU on Windows. Thanks.

https://github.com/mattn/go-tflite/blob/master/_example/ssd_edgetpu/main.go

FYI: latst master branch seems not work for edgecpu.

If you are on Windows, try to fix Makefile for d855adfc5a0195788bf5f92c3c7352e638aa1109. (This is already fixed in master branch of TensorFlow, but master branch does not work eventhough applying this patch)

diff --git a/tensorflow/lite/tools/make/Makefile b/tensorflow/lite/tools/make/Makefile
index 4219aa5c31..e7611bacfe 100644
--- a/tensorflow/lite/tools/make/Makefile
+++ b/tensorflow/lite/tools/make/Makefile
@@ -102,6 +102,7 @@ $(wildcard tensorflow/lite/*.c) \
 $(wildcard tensorflow/lite/c/*.c) \
 $(wildcard tensorflow/lite/core/*.cc) \
 $(wildcard tensorflow/lite/core/api/*.cc) \
+$(wildcard tensorflow/lite/tools/optimize/sparsity/*.cc) \
 $(wildcard tensorflow/lite/experimental/resource_variable/*.cc) \
 $(wildcard tensorflow/lite/experimental/ruy/*.cc)
 ifneq ($(BUILD_TYPE),micro)

[mattn]

I hope you to fix to work on master branch of TensorFlow but if you want to close this issue, feel free to close.

[Namburger]

@mattn yes the tflite_runtime-2.1.0.post1 package as well as all of our cpp lib for the current release is build on top of d855adfc5a0195788bf5f92c3c7352e638aa1109. The fix was merged in after this pull request which is a little bit later, so it's not included in d855adfc5a0195788bf5f92c3c7352e638aa1109, unfortunately.
Also, very interesting project, not sure if you've seen doods, it uses your go wrapper :)

[mattn]

I found doods 2 hours ago. :)

[Namburger]

@mattn I guess this is not even related but inspired by doods, there is restor, now we are full cycle :)

[mattn]

Well, then edgecpu can be used with latest tensorflow?

[Namburger]

@mattn unfortunately no, but I closed because this can't be fixed due to libedgetpu being closed source. Next release is going to be compatible with newer tensorflow commit.
FYI we are in the talk for open sourcing the libraries, at that time all these issues with go away!

[snowzach]

Oh Hai! I wrote doods. I just ran into this issue also. I was trying to compile on top of TF v1.15.3 and am having the same issue.

So a quick question, I had read somewhere that TF 2 was slower than TF 1 so I've still been using 1. Should I just use that commit above? Will it slow down detection? (I guess I could just try it and see)

[snowzach]

@Namburger do you have a timeline for the next release that is going to be compatible with newer Tensorflow? I cross-compile DOODS for arm32 and arm64 and getting that to work is already a nightmare. I haven't been able to get the edgetpu compatible commit to be able to cross-compile on these platforms yet. Just curious if you know when that might happen.

[Namburger]

@snowzach

I had read somewhere that TF 2 was slower than TF 1

Humn, I actually never heard of this, do you have some reference?

do you have a timeline for the next release that is going to be compatible with newer Tensorflow?

No timeline on an official release, unfortunately :/
However, libedgetpu just became open sourced! So in theory you can 1) build libedgetpu 2) build tensorflow-lite.a on the same commit and things should works in harmony. Tip of this libedgetpu repo is currently on f394a768 which is much newer than the previous release. Here is a quick guide for doing this on an x86 machine with a USB accelerator. I haven't tested the cross compilation yet, but you should be able to do make CPU=aarch64 to produce libedgetpu for the devboard (build was successful for me, just haven't tested).

[snowzach]

Hey @Namburger if you google tensorflow 2 is slower than 1 it comes up with tons of results. This might be related to training though. I actually did just run a test today to compare and as far as DOODS is concerned, the speed seems to be the same so I will keep moving forward with the latest.

Do I need to compile the edgetpu library first? I'm not so great with other build tools besides make. I will try to follow that example. Thanks!

Edit: I am getting this Unrecognized option: --experimental_repo_remote_exec when I try to compile. (Nevermind, I guess I need to upgrade bazel)

[Namburger]

@snowzach o_0, I guess I must have been living under a rock all these time haha

Here is a super duper minimal build with just make file (I prebuilt libedgetpu, libtensorflow + some necessary includes), you're welcome to examine it.

tar xvf edgetpu_build.tar.gz && cd edgetpu_build
make
LD_LIBRARY_PATH=$(pwd)/lib ./main

[Namburger]

ahhh sorry, forgot to share the link :)
https://drive.google.com/file/d/1nsbOQtZIid3NCDjdHHob72r4YDSU2Qud/view?usp=sharing

[snowzach]

Will check it out. Thanks!

[snowzach]

Awesome thanks @Namburger DOODS is now using the open source library!

[Namburger]

Nice!

[Namburger]

@mattn

Well, then edgecpu can be used with latest tensorflow?

I guess I forgot to mention to you that libedgetpu is open source now.
You can build it from any tensorflow commit you want :)

[mattn]

Wow!