Edgetpu: using edgetpu c++ api crash

[akioolin]

Dear Sir:

After some code trying and library integration, finally I get a runable code sequence. the code is as the following. the code uses openCV for video capture, format conversion, resize, etc. the edgetpu part is using edgetpu c++ api which found in src/cpp. the crash log is got by setting "edgetpu::EdgeTpuManager::GetSingleton()->SetVerbosity(10);"

the teset code is the main.edgetpu_api.log. please rename to main.edgetpu_api.cpp
main.edgetpu_api.log

the crash log is as the following file.
edgetpu crash.log

Could any one give me some advice? Thanks alot.

BR, Akio

[Namburger]

@akioolin any way you can also provide a make file to build this?
Otherwise, just looking at the log, I can't see any indication of it crashing before it gets cut short :/
What message was displayed on the console when the crash occur?
Comparing the log from your cout statement, it didn't looks like it got far at all since I couldn't even find this line (correct me if I'm wrong):

std::cout << "input tensor dimension:" << dim_size << "\r\n";

additionally, are you able to run any of our cpp examples?

[akioolin]

@Namburger Thank you and I'm sorry for the incomplete log provided. I had put the whole project in https://github.com/akioolin/edgetpu_demo

there is two parts. the first one is edgetpu_api, which extract from edgetpu/src/cpp, and form a static library, which been used in the second project, edgetpu_demo.
the second project is edgetpu_demo which is targeted as a C++ based demo application, using the first project's library, libedgetpu_api.a and OpenCV.

the log is extract from the edgetpu_demo's outcome message. and I'd removed the cout output message, just keep the log which come from edgetpu related library.
If it is not a code related problem, it seems a power adapter output current problem.
If so, I'll do using power supply to make sure how much current for running SSD model.

If you have any problem, please let me know.

BR, Akio

[akioolin]

@Namburger BTW, the following script will install the needed package.
sudo apt install mc joe samba libhdf5-103 libqtgui4 libatlas-base-dev libqt4-test python3-opencv libopencv-dev astyle libncurses-dev ntpdate git cmake

[Namburger]

@akioolin
is it possible that the toolchain you are using is missing by mistake? So far I've installed all the deps and make in the edgetpu_demo dir:

g++ -Wall -std=c++11 -O2 -mcpu=cortex-a53 -march=armv8-a -I. -I./include -I./include/edgetpu -I./include/src/cpp -I./include/src/cpp/examples -DVERSION_STR=\"1.0\"  -o main.o -c main.cpp
g++: warning: ‘-mcpu=’ is deprecated; use ‘-mtune=’ or ‘-march=’ instead
cc1plus: error: bad value (‘armv8-a’) for ‘-march=’ switch
cc1plus: note: valid arguments to ‘-march=’ switch are: nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 bonnell atom silvermont slm knl x86-64 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 btver1 btver2
cc1plus: error: bad value (‘cortex-a53’) for ‘-mtune=’ switch
cc1plus: note: valid arguments to ‘-mtune=’ switch are: nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 bonnell atom silvermont slm knl intel x86-64 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 btver1 btver2 generic; did you mean ‘corei7-avx’?
Makefile:67: recipe for target 'main.o' failed
make: [main.o] Error 1 (ignored)
g++ -Wall  -O2 -mcpu=cortex-a53 -march=armv8-a -std=c++11 -o edgetpu_demo main.o   -Wl,-rpath,/usr/local/lib  -L./lib -L/lib64/aarch64-linux-gnu -L/usr/lib/aarch64-linux-gnu -lpthread -lm -ldl -lusb-1.0 -lglog -ledgetpu -ledgetpu_api -lopencv_core -lopencv_imgproc -lopencv_imgcodecs  -lopencv_highgui -lopencv_videoio -ltensorflow-lite -labsl_synchronization -labsl_base -labsl_spinlock_wait -labsl_malloc_internal -labsl_symbolize -labsl_stacktrace -labsl_debugging_internal -labsl_dynamic_annotations -labsl_demangle_internal -labsl_time -labsl_time_zone -labsl_graphcycles_internal -labsl_int128
g++: warning: ‘-mcpu=’ is deprecated; use ‘-mtune=’ or ‘-march=’ instead
g++: error: main.o: No such file or directory
Makefile:61: recipe for target 'edgetpu_demo' failed
make: [edgetpu_demo] Error 1 (ignored)

Initially, I mistakenly thought that you are using a USB accelerator. This looks like you are compiling for the dev board, power could be an issue, that power supply are you using?

[akioolin]

@Namburger The project I put on github is based on EdgeTPU Dev-Board. please use a Dev-Board to test the code. Sorry for this part. not very clear point to the suitable test target.
Yes, I use Dev-Board for my develop target. the recommended power adapter's current is 3A above or 4A above?

BR, Akio

[Namburger]

@akioolin
3A@5V should be okay on the dev board. Are you building the project on the dev board also? Right now I'm still having problem building the code. I don't have a board with me, trying to build this on my laptop, which is correct right?

Anyways, is this missing on purpose:
https://github.com/akioolin/edgetpu_demo/blob/master/edgetpu_demo/Makefile#L8
I supplied added my own toolchain here:

http://releases.linaro.org/components/toolchain/binaries/7.5-2019.12/aarch64-linux-gnu/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz

and now compiling I'm getting this:

gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ -Wall -std=c++11 -O2 -mcpu=cortex-a53 -march=armv8-a -I. -I./include -I./include/edgetpu -I./include/src/cpp -I./include/src/cpp/examples -I/usr/include -I./gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include -DVERSION_STR=\"1.0\"  -o main.o -c main.cpp
In file included from /home/nam/edgetpu_demo/edgetpu_demo/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/aarch64-linux-gnu/include/c++/7.5.0/cstdint:41:0,
                 from /usr/include/opencv2/core/hal/interface.h:34,
                 from /usr/include/opencv2/core/cvdef.h:67,
                 from /usr/include/opencv2/core.hpp:52,
                 from /usr/include/opencv2/opencv.hpp:52,
                 from main.cpp:7:
/usr/include/stdint.h:34:10: fatal error: bits/stdint-intn.h: No such file or directory
 #include <bits/stdint-intn.h>
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:69: recipe for target 'main.o' failed
make: *** [main.o] Error 1

Any idea :/
what toolchain are you using?

[akioolin]

@Namburger I build the whole project on EdgeTPU Dev-Board. the cross compiler which I used is linaro-7.4.1.
You could find the toolchain in https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/

for bits/stdint-intn.h error, I didn't meet this problem. in dev-board, the file is in /usr/include/aarch64-linux-gnu/bits. in my experience, the problem should be toolchain installation problem.

For the power problem, I just do a test to check the power consumption during edgetpu working. the current is about 0.92A. But there is no HDMI display. the log is HDMI display attached to dev-board. I'll check more detail about HW configuration.

BR, Akio

[Namburger]

@akioolin
So after successfully compiling the code on the dev board with some extra dependencies installed, the only crash I encountered was due to this:

VIDEOIO ERROR: V4L: index 1 is not correct!
Open Camera Failed!!

This was due to this line:

cv::VideoCapture uvc_camera(1);

Which I changed to

cv::VideoCapture uvc_camera(0);

as the default coral's camera is in /dev/video0. Although this could change if you are using a usb camera. But otherwise, I believe the code works as expected as I'm not able to reproduce a crash:

mendel@bored-calf:~/edgetpu_demo/edgetpu_demo$ ./edgetpu_demo 
input tensor dimension:4
input tensor batch:1
input tensor width:300
input tensor height:300
input tensor channel:3
output tensor dimension:4
output tensor batch:80
output tensor width:20
output tensor height:20
output tensor channel:1
Open Camera Done.
Camera frame width: 640
Camera frame height: 480
Camera frame fps: 30
no object detect
object=> 0  person, probability=> 0.386719
bottom=> 0
left=> 0.003021
top=> 0.996909
right=> 1
object=> 0  person, probability=> 0.386719
bottom=> 0
left=> 0.0115965
top=> 0.996909
right=> 0.993793
object=> 80  sink, probability=> 0.472656
bottom=> 0.014375
left=> 0.014375
top=> 0.962975
right=> 0.962975

Although this doesn't mean that I'm getting the same error as you. What is the actual output that you see before the crash if you turn off verbosity and keep the cout lines?

[akioolin]

@Namburger Thank you very much. the output is correct. if I turn off the verbosity, the output will as the same as you see. I'll attach a current meter to see the current change to clarify the power issue.

BR, Akio

[akioolin]

@Namburger After some tests, I found the power adapter is broken. The output current is not enough for running object detection. By using a simple current meter with a new power adapter to monitor current changes, I got the following summery.

for boot up sequence, max current ~1.20A
for realtime object detection no hdmi display output, max current ~0.95A
for realtime object detection with hdmi display output, max current ~1.45A
if the fan spin, the current will plus 0.15A

BR, Akio

[akioolin]

@Namburger

After Some tests, I found a very strange stuff. Please take a look of my latest update code, main.cpp in github. The label index of detect result should be in 0~79, from the model's output tensor dimension and the respect label file's classes is also 80 classes. But the detection outcome will be greater than 80.

The normal output is like this.
object[0]=> cat_index=> 61, name:69 toilet, probability=> 0.730469, (295, 177)-(450, 380)
object[1]=> cat_index=> 0, name:0 person, probability=> 0.730469, (0, 64)-(372, 471)
object[2]=> cat_index=> 78, name:88 hair drier, probability=> 0.417969, (538, 269)-(636, 452)

The strange output is like this.
object[0]=> cat_index=> 0, name:0 person, probability=> 0.582031, (0, 69)-(380, 478)
object[1]=> cat_index=> 61, name:69 toilet, probability=> 0.5, (293, 179)-(445, 363)
object[2]=> cat_index=> 83, name:, probability=> 0.417969, (309, 104)-(330, 138)

Due to category index is greater than 80, there is no respect label name could be shown.

Could you do me a favor to give me some idea to check where is the missing part?

Many Thanks for your help.

BR, Akio

[Namburger]

@akioolin
If you look at the label file, some of the numbers are skipped.
So while wc -l etc/label/mobilenet_ssd_v1_coco_quant_postprocess_edgetpu.label only returns 79 lines, some of the indexes are skipped! If you got this label file from our coco_labels.txt, you can take a look at this answer:
https://github.com/google-coral/edgetpu/issues/29#issuecomment-558273877

[akioolin]

@Namburger Thank you.

So I have to use the cat_index to find the correct class label name?

BR, Akio

[Namburger]

@akioolin No, what you are doing is perfectly fine, the index matches exactly that's in your label file!

[akioolin]

@Namburger Thank you. BTW, Is there any should be modify in the code? this project will be come about 3 prototype product base.

BR, Akio

[Namburger]

@akioolin Everything looks good!

[akioolin]

@Namburger Thank you very much.

Hope the prototype will be done before End of Q3.

BR, Akio