Caffe: make pycaffe: numpy/arrayobject.h: No such file or directory

[martyychang]

What steps reproduce the bug?

Following the Ubuntu Installation instructions...

1. Launch a new g2.2xlarge instance on AWS
2. Install the NVIDIA driver
3. Install CUDA 7.5
4. Follow steps below to clone the repo and build the RC3 release.

# Git clone BVLC/caffe
sudo apt-get update
sudo apt-get install -y git
mkdir Workspace && cd Workspace
git clone https://github.com/BVLC/caffe.git
cd caffe && git checkout -b rc3-build rc3

# Caffe RC3 with ATLAS
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
sudo apt-get install libatlas-base-dev  # ATLAS

cd ~/Workspace/caffe && cp Makefile.config.example Makefile.config
make all  # -j8
make test
make runtest

# PyCaffe
sudo apt-get install -y python-dev
make pycaffe

At this point compilation fails with the message, "numpy/arrayobject.h: No such file or directory".

Can you reproduce the bug using the latest master, compiled with the DEBUG make option? Yes

What hardware and operating system/distribution are you running? AWS g2.2xlarge instance running Ubuntu Server 14.04 LTS (ami-2d39803a)

[martyychang]

I resolved this by installing Anaconda and then editing Makefile.config. I think it would be great if the docs could be updated with an Anaconda-specific installation, which could accelerate adoption of Caffe.

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
                /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
                  $(ANACONDA_HOME)/include/python2.7 \
                  $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

[crowsonkb]

You need to find where the numpy headers _actually_ are and edit Makefile.config. For instance I have python3.5 installed via homebrew on OS X and numpy installed with pip, without a virtualenv. So my numpy include files are in /usr/local/lib/python3.5/site-packages/numpy/core/include. If you have numpy installed inside a virtualenv you'll have to go hunting around in it for the numpy/core/include directory. Otherwise check to make sure your Makefile.config isn't confusing site-packages and dist-packages.

[codeislife99]

1. The right solution is to locate the directory where your numpy is installed. You can do this by running
   python -m site.
2. Check all of the path/to/site-packages or path/to/dist-packages directories. Check which one of them contains numpy.
3. Then change the numpy path in PYTHON_INCLUDE := earlierpath/numpy/core/include in Makefile.config.
4. Now make clean & make all
5. This will always work since it will always find numpy.