Caffe: make runtest failed

[kkarnatak]

I have the following error while running the 'make runtest'. I have tried with cpu flag enabled and disabled both, same error.

----------] 5 tests from ImageDataLayerTest/1, where TypeParam = caffe::CPUDevice<double> [ RUN ] ImageDataLayerTest/1.TestReshape [ OK ] ImageDataLayerTest/1.TestReshape (30 ms) [ RUN ] ImageDataLayerTest/1.TestShuffle [ OK ] ImageDataLayerTest/1.TestShuffle (113 ms) [ RUN ] ImageDataLayerTest/1.TestRead [ OK ] ImageDataLayerTest/1.TestRead (114 ms) [ RUN ] ImageDataLayerTest/1.TestResize *** Aborted at 1481237980 (unix time) try "date -d @1481237980" if you are using GNU date *** PC: @ 0x7ffb7938b0ba (unknown) *** SIGSEGV (@0xfffffffffffffff7) received by PID 5373 (TID 0x7ffb62eaa700) from PID 18446744073709551607; stack trace: *** @ 0x7ffb8254d670 (unknown) @ 0x7ffb7938b0ba (unknown) @ 0x7ffb7938b18b (unknown) @ 0x7ffb7938cce8 (unknown) @ 0x7ffb7938b692 (unknown) @ 0x7ffb79386020 (unknown) @ 0x7ffb79384165 tbb::internal::allocate_root_with_context_proxy::allocate() @ 0x7ffb80db6e22 cv::parallel_for_() @ 0x7ffb8117ab2a (unknown) @ 0x7ffb81177bb7 cv::resize() @ 0x7ffb83061997 caffe::ReadImageToCVMat() @ 0x7ffb82f708bb caffe::ImageDataLayer<>::load_batch() @ 0x7ffb82efc906 caffe::BasePrefetchingDataLayer<>::InternalThreadEntry() @ 0x7ffb82984596 (unknown) @ 0x7ffb8254370a start_thread @ 0x7ffb801b40af clone @ 0x0 (unknown) Segmentation fault (core dumped) src/caffe/test/CMakeFiles/runtest.dir/build.make:57: recipe for target 'src/caffe/test/CMakeFiles/runtest' failed

[cdluminate]

What's your operating system? Ubuntu devel seems to suffer from a similar problem.

https://launchpadlibrarian.net/298212926/buildlog_ubuntu-zesty-amd64.caffe_1.0.0~rc3+20161127-g24d2f67-4_BUILDING.txt.gz

[kkarnatak]

I am using kubuntu 16.10

On 16 Dec 2016 02:18, "L. Zhou" [e-mail] wrote:

What's your operating system? Ubuntu devel seems to suffer from a similar
problem.

https://launchpadlibrarian.net/298212926/buildlog_ubuntu-
zesty-amd64.caffe_1.0.0~rc3+20161127-g24d2f67-4_BUILDING.txt.gz
https://launchpadlibrarian.net/298212926/buildlog_ubuntu-zesty-amd64.caffe_1.0.0%7Erc3+20161127-g24d2f67-4_BUILDING.txt.gz

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/BVLC/caffe/issues/5077#issuecomment-267492285, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH_dXyuJKIfFjQfHQKQklEM7PubSEYf1ks5rIebQgaJpZM4LIYwD
.

[stjordanis]

I also have ubuntu 16.10, no cpu.

I have the same issue too, although:

• Pycaffe runs ok
• lenet runs ok

After I set DEBUG=1, and I ran:

protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto

because I had issues with caffe.pb.h,

runtest fails after a while when running:
make runtest

I get (excluding some needless lines):
.build_debug/tools/caffe
.build_debug/test/test_all.testbin 0 --gtest_shuffle --gtest_filter="-GPU"
Note: Google Test filter = -GPU
Note: Randomizing tests' orders with a seed of 77085 .

Many test pass but then:
[----------] 5 tests from ImageDataLayerTest/1, where TypeParam = caffe::CPUDevice
[ RUN ] ImageDataLayerTest/1.TestRead
[ OK ] ImageDataLayerTest/1.TestRead (84 ms)
[ RUN ] ImageDataLayerTest/1.TestResize
* Aborted at 1481935191 (unix time) try "date -d @1481935191" if you are using GNU date
PC: @ 0x7f14a0c870ba (unknown)
SIGSEGV (@0xfffffffffffffff7) received by PID 16487 (TID 0x7f14871f2700) from PID 18446744073709551607; stack trace: *
@ 0x7f14a2152630 (unknown)
@ 0x7f14a0c870ba (unknown)
@ 0x7f14a0c8718b (unknown)
@ 0x7f14a0c88ce8 (unknown)
@ 0x7f14a0c87692 (unknown)
@ 0x7f14a0c82020 (unknown)
@ 0x7f14a0c80165 tbb::internal::allocate_root_with_context_proxy::allocate()
@ 0x7f14a3e50e22 cv::parallel_for_()
@ 0x7f14a3737b2a (unknown)
@ 0x7f14a3734bb7 cv::resize()
@ 0x7f14a2c093dd caffe::ReadImageToCVMat()
@ 0x7f14a2d0694c caffe::ImageDataLayer<>::load_batch()
@ 0x7f14a2c9037c caffe::BasePrefetchingDataLayer<>::InternalThreadEntry()
@ 0x7f14a2c2e19b caffe::InternalThread::entry()
@ 0x7f14a2c307d1 boost::_mfi::mf5<>::operator()()
@ 0x7f14a2c306a9 boost::_bi::list6<>::operator()<>()
@ 0x7f14a2c3059e boost::_bi::bind_t<>::operator()()
@ 0x7f14a2c30550 boost::detail::thread_data<>::run()
@ 0x7f14a33f7596 (unknown)
@ 0x7f14a21486ca start_thread
@ 0x7f14a1e820af clone
@ 0x0 (unknown)
Segmentation fault (core dumped)

~~~~~~~~~~
My config file:

USE_CUDNN := 1

CPU-only switch (uncomment to build without GPU support).

CPU_ONLY := 1

uncomment to disable IO dependencies and corresponding data layers

#USE_OPENCV := 1
#USE_LEVELDB := 1
#USE_LMDB := 0

ALLOW_LMDB_NOLOCK := 1

Uncomment if you're using OpenCV 3

OPENCV_VERSION := 3

To customize your choice of compiler, uncomment and set the following.

N.B. the default for Linux is g++ and the default for OSX is clang++

CUSTOM_CXX := g++

CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

On Ubuntu 14.04, if cuda tools are installed via

"sudo apt-get install nvidia-cuda-toolkit" then use this instead:

CUDA_DIR := /usr

CUDA architecture setting: going with all of them.

For CUDA < 6.0, comment the *_50 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50

BLAS choice:

atlas for ATLAS (default)

mkl for MKL

open for OpenBlas

BLAS := atlas

Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

Leave commented to accept the defaults for your choice of BLAS

(which should work)!

BLAS_INCLUDE := /path/to/your/blas

BLAS_LIB := /path/to/your/blas

Homebrew puts openblas in a directory that is not on the standard search path

BLAS_INCLUDE := $(shell brew --prefix openblas)/include

BLAS_LIB := $(shell brew --prefix openblas)/lib

This is required only if you will compile the matlab interface.

MATLAB directory should contain the mex binary in /bin.

MATLAB_DIR := /usr/local

MATLAB_DIR := /Applications/MATLAB_R2012b.app

NOTE: this is required only if you will compile the python interface.

We need to be able to find Python.h and numpy/arrayobject.h.

#PYTHON_INCLUDE := /usr/include/python2.7 \

/usr/lib/python2.7/dist-packages/numpy/core/include

Anaconda Python distribution is quite popular. Include path:

Verify anaconda location, sometimes it's in root.

ANACONDA_HOME := $(HOME)/anaconda

PYTHON_INCLUDE := /home/steliox/anaconda2/include/python2.7 \
/home/steliox/anaconda2/include \
/home/steliox/anaconda2/lib/python2.7/site-packages/numpy/core/include \

Uncomment to use Python 3 (default is Python 2)

PYTHON_LIBRARIES := boost_python3 python3.5m

# PYTHON_INCLUDE := /usr/include/python3.5m \

/usr/lib/python3.5/dist-packages/numpy/core/include

We need to be able to find libpythonX.X.so or .dylib.

PYTHON_LIB := /usr/lib

PYTHON_LIB := /home/steliox/anaconda2/lib

Homebrew installs numpy in a non standard path (keg only)

PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include

PYTHON_LIB += $(shell brew --prefix numpy)/lib

Uncomment to support layers written in Python (will link against Python libs)

WITH_PYTHON_LAYER := 1

Whatever else you find you need goes here.

INCLUDE_DIRS := /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/hdf5/serial/

If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

INCLUDE_DIRS += $(shell brew --prefix)/include

LIBRARY_DIRS += $(shell brew --prefix)/lib

Uncomment to use pkg-config to specify OpenCV library paths.

(Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

USE_PKG_CONFIG := 1

N.B. both build and distribute dirs are cleared on make clean

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171

DEBUG := 1

The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

enable pretty build (comment to see full commands)

Q ?= @

[mmderakhshani]

Hi,
I would like to know, Is your problem solved? when I run make runset,I have got a same issue on ubuntu 16.04! could you please help me?

[stjordanis]

@mohammad-py It still fails in the same way exactly as mentioned above...
I just cloned the latest caffe git repo + I have been updating my ubuntu system normally, running v17 now:
I made the same procedure, but....
when I tried
make runtest
the same things happen.

[svanschalkwyk]

I get Check failed: error == cudaSuccess (35 vs. 0)
CUDA driver version is insufficient for CUDA runtime version
Ubuntu 16.10 (Mint), gcc5, CUDA 8, Driver 378

[rongfeng-china]

I installed boost1.58 and added linked library and include path in bashrc.
Then I had an error running ‘make runtest', and finally I found that I made a tiny mistake in bashrc.

The boost lib should be added to LD_LIBRARY, rather than LIBRARY...

[kkarnatak]

Hi Guys,

I started using https://github.com/BVLC/caffe/tree/opencl

and now I am able to successfully execute make runtest. I am using opencl mode. I havent tried CPU only mode but if you have AMD this will work provided you have successfully installed AMD graphic card. I had to downgrade to 14.04 to install AMD graphic card.

[shelhamer]

From https://github.com/BVLC/caffe/blob/master/CONTRIBUTING.md:

_Please do not post usage, installation, or modeling questions, or other requests for help to Issues._
Use the caffe-users list instead. This helps developers maintain a clear, uncluttered, and efficient view of the state of Caffe.

[fan0916]

hello all
i have issued this today, and finally i've solved this by changing libtbb2 version
i'm using Ubuntu 16.04 and it uses libtbb2 (4.4~20160526-0ubuntu1) [universe] as default
from belowlink i found that there is a bug in pre-build package
https://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/636519
so i changed to a new libtbb2 package as follow:
libtbb2 (4.4~20151115-0ubuntu3) [universe] http://packages.ubuntu.com/xenial/libtbb2
this solved my problem
regards!