Caffe: Caffe make runtest error

Created on 3 Oct 2017 · 5Comments · Source: BVLC/caffe

Issue summary

I try to compile caffe, when I make runtest, there are some errors.

[       OK ] NetTest/1.TestFromTo (1 ms)
[ RUN      ] NetTest/1.TestHasLayer
[       OK ] NetTest/1.TestHasLayer (0 ms)
[ RUN      ] NetTest/1.TestHasBlob
[       OK ] NetTest/1.TestHasBlob (1 ms)
[ RUN      ] NetTest/1.TestBackwardWithAccuracyLayer
[       OK ] NetTest/1.TestBackwardWithAccuracyLayer (1 ms)
[ RUN      ] NetTest/1.TestAllInOneNetDeploy
*** Error in `.build_release/test/test_all.testbin': free(): invalid pointer: 0x00007fdfa0c1e9c8 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fdfddfd67e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fdfddfdf37a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fdfddfe353c]
.build_release/test/test_all.testbin[0x52592c]
.build_release/test/test_all.testbin[0x785a68]
.build_release/test/test_all.testbin[0x731bf0]
.build_release/test/test_all.testbin[0x78328a]
.build_release/test/test_all.testbin[0x920273]
.build_release/test/test_all.testbin[0x91988a]
.build_release/test/test_all.testbin[0x9199d8]
.build_release/test/test_all.testbin[0x919ab5]
.build_release/test/test_all.testbin[0x91ad8f]
.build_release/test/test_all.testbin[0x91b0b3]
.build_release/test/test_all.testbin[0x46ebcd]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fdfddf7f830]
.build_release/test/test_all.testbin[0x4766e9]
======= Memory map: ========
00400000-00af5000 r-xp 00000000 08:11 24516491                           /home/bian/DL/caffe/.build_release/test/test_all.testbin
00cf4000-00d58000 r--p 006f4000 08:11 24516491                           /home/bian/DL/caffe/.build_release/test/test_all.testbin
00d58000-00d5a000 rw-p 00758000 08:11 24516491                           /home/bian/DL/caffe/.build_release/test/test_all.testbin
0144b000-0ffe0000 rw-p 00000000 00:00 0                                  [heap]
10000000-10001000 rw-s 00000000 00:06 20400                              /dev/nvidia0
10001000-10002000 rw-s 00000000 00:06 20400                              /dev/nvidia0
10002000-10003000 rw-s 00000000 00:06 20400                              /dev/nvidia0
...............
7fdfe7618000-7fdfe7619000 rw-s 00000000 00:06 20399                      /dev/nvidiactl
7fdfe7619000-7fdfe761a000 rw-s 00000000 00:06 20399                      /dev/nvidiactl
7fdfe761a000-7fdfe761b000 rw-s 00000000 00:05 13140008                   /SYSV00510510 (deleted)
7fdfe761b000-7fdfe761c000 rw-s 00000000 00:05 13107238                   /SYSV00000000 (deleted)
7fdfe761c000-7fdfe761f000 rw-p 00000000 00:00 0 
7fdfe761f000-7fdfe7620000 r--p 00025000 08:11 11276709                   /lib/x86_64-linux-gnu/ld-2.23.so
7fdfe7620000-7fdfe7621000 rw-p 00026000 08:11 11276709                   /lib/x86_64-linux-gnu/ld-2.23.so
7fdfe7621000-7fdfe7622000 rw-p 00000000 00:00 0 
7ffe842ec000-7ffe8430c000 rwxp 00000000 00:00 0                          [stack]
7ffe8430c000-7ffe8430e000 rw-p 00000000 00:00 0 
7ffe84326000-7ffe84328000 r--p 00000000 00:00 0                          [vvar]
7ffe84328000-7ffe8432a000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
*** Aborted at 1507009506 (unix time) try "date -d @1507009506" if you are using GNU date ***
PC: @     0x7fdfddf94428 gsignal
*** SIGABRT (@0x3e800005bfc) received by PID 23548 (TID 0x7fdfe75e6740) from PID 23548; stack trace: ***
    @     0x7fdfde33a390 (unknown)
    @     0x7fdfddf94428 gsignal
    @     0x7fdfddf9602a abort
    @     0x7fdfddfd67ea (unknown)
    @     0x7fdfddfdf37a (unknown)
    @     0x7fdfddfe353c cfree
    @           0x52592c caffe::MakeTempFilename()
    @           0x785a68 caffe::NetTest<>::InitNetFromProtoFileWithState()
    @           0x731bf0 caffe::NetTest<>::InitAllInOneNet()
    @           0x78328a caffe::NetTest_TestAllInOneNetDeploy_Test<>::TestBody()
    @           0x920273 testing::internal::HandleExceptionsInMethodIfSupported<>()
    @           0x91988a testing::Test::Run()
    @           0x9199d8 testing::TestInfo::Run()
    @           0x919ab5 testing::TestCase::Run()
    @           0x91ad8f testing::internal::UnitTestImpl::RunAllTests()
    @           0x91b0b3 testing::UnitTest::Run()
    @           0x46ebcd main
    @     0x7fdfddf7f830 __libc_start_main
    @           0x4766e9 _start
    @                0x0 (unknown)
Makefile:532: recipe for target 'runtest' failed
make: *** [runtest] Aborted (core dumped)

Steps to reproduce

In Makefile.config

LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda/lib

USE_CUDNN := 1

OPENCV_VERSION := 3

CUDA_ARCH :=    -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_52,code=sm_52 \

BLAS := open

ANACONDA_HOME := $(HOME)/anaconda

PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := $(ANACONDA_HOME)/include/python3.6m \
                 $(ANACONDA_HOME)/lib/python3.6/dist-packages/numpy/core/include
PYTHON_LIB := $(ANACONDA_HOME)/lib

WITH_PYTHON_LAYER := 1

INCLUDE_DIRS := /usr/include/hdf5/serial $(PYTHON_INCLUDE) /usr/local/include $(ANACONDA_HOME)/include
LIBRARY_DIRS := /usr/lib/x86_64-linux-gnu/hdf5/serial $(PYTHON_LIB) /usr/local/lib /usr/lib

In Makefile,
replace this line:
NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
with the following line
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

edit the file \ After the line 89, add the new line that contains the following:
template class BlockingQueue<Datum*>;

Your system configuration

Operating system: Ubuntu16.04
Compiler: g++ 5.4 gcc 5.4
CUDA version (if applicable): 8.0.61.2
CUDNN version (if applicable): 7
BLAS: OpenBLAS
Python or MATLAB version (for pycaffe and matcaffe respectively): Anaconda Python 3.6

Source