Edgetpu: Coral Usb Accelerator not working with TensorFlow Lite C++ API

[Fcsalvagnini]

I bought a Coral Usb Accelerator for deploying purposes. When I try to use it, with the TensorFlow Lite API in C++, it throws the following error:

ERROR: Internal: Unsupported data type in custom op handler: 10812496
ERROR: Node number 0 (edgetpu-custom-op) filed to prepare.

Failed to allocate tensors.
Segmentation fault (core dumped)

I'm using a model compiled for the edgetpu device (mobilenet_ssd_v1_coco_quant_postprocess_edgetpu.tflite). I tried to use the same model with the tflite_runtime for python 3.5 and everything goes right, with the Object Detection being carried out in the Coral USB Accelerator.

[Namburger]

Hey, make sure you build your libtflite from this commit

[Fcsalvagnini]

By libtflite do you reffer to the libtensorflow-lite.a?

[Namburger]

@Fcsalvagnini yes, sorry for the confusion.
With the current libedgetpu.so runtime (13) we build everything with that commit of tensorflow.

[Fcsalvagnini]

@Namburger All right, I'll compile it again from that specif commit and after that, I return to you. Just a sec...

[Fcsalvagnini]

@Namburger Now at the end of the compiling process, when the benchmark is carried out, I got the following error:

g++ -O3 -DNDEBUG -fPIC  --std=c++11 -fPIC -DGEMMLOWP_ALLOW_SLOW_SCALAR_FALLBACK -pthread -I. -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/../../../../../ -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/../../../../../../ -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/ -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/eigen -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/absl -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/gemmlowp -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/neon_2_sse -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/farmhash/src -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/flatbuffers/include -I/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/downloads/fp16/include -I -I/usr/local/include \
-o /home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/bin/benchmark_model_performance_options /home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/obj/tensorflow/lite/tools/benchmark/benchmark_tflite_performance_options_main.o \
 /home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/lib/benchmark-lib.a  -lstdc++ -lpthread -lm -lz -ldl
/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/lib/libtensorflow-lite.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x3a2): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x3ad): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x682): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x68d): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'
collect2: error: ld returned 1 exit status
tensorflow/lite/tools/make/Makefile:286: recipe for target '/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/bin/minimal' failed
make: *** [/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/bin/minimal] Error 1
make: *** Waiting for unfinished jobs....
/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/lib/benchmark-lib.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x3a2): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x3ad): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x682): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x68d): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'
collect2: error: ld returned 1 exit status
tensorflow/lite/tools/make/Makefile:300: recipe for target '/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/bin/benchmark_model' failed
make: *** [/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/bin/benchmark_model] Error 1
/home/ct1/Downloads/tflite/third-party/tensorflow/tensorflow/lite/tools/make/gen/linux_x86_64/lib/benchmark-lib.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x3a2): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x3ad): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x682): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x68d): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'
collect2: error: ld returned 1 exit status

Nevertheless, a libtensorflow-lite.a was generated, but when I try to use It I got a similar error:

/home/ct1/Downloads/tflite/lib/libtensorflow-lite.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x3a2): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x3ad): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x682): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x68d): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'
collect2: error: ld returned 1 exit status

[Namburger]

Hello, ahh, so another issue is that the specific commit: https://github.com/tensorflow/tensorflow/commit/d855adfc5a0195788bf5f92c3c7352e638aa1109 also has a build bug which is fixed in this commit: https://github.com/tensorflow/tensorflow/commit/72cd947f231950d7ecd1406b5a67388fef7133ea

So in order to get it working, you need to first remove old build and checkout this commit: https://github.com/tensorflow/tensorflow/commit/d855adfc5a0195788bf5f92c3c7352e, add this line and then rebuild.

Overall, the quickest way to do this is to just use our provided make file in this repo, which you can just run:

make CPU=<your-choice> examples

or

make DOCKER_IMAGE=debian:buster DOCKER_CPUS=<your-choice> DOCKER_TARGETS=examples docker-build

[Fcsalvagnini]

Ok. Thank you so much @Namburger . I made the changes and my application runs perfectly.

[Namburger]

@Fcsalvagnini Awesome, definitely sorry for the confusing issue, I'll ping our documentation team to get this better documented.

[Namburger]

@Fcsalvagnini FYI: Here is a single cmake example build file with all dependencies: https://github.com/google-coral/edgetpu/issues/54#issuecomment-616149916

[Fcsalvagnini]

@Fcsalvagnini FYI: Here is a single cmake example build file with all dependencies: #54 (comment)

Thank you. That's good to know!