Dlib: Ubuntu, pip3 install dlib (compile error: can not find "Python.h")

[kilho]

Expected Behavior

installing only python dlib in ubuntu using
pip3 install dlib

I think I have installed all the requirements including python3-dev. And, I see the "Python.h" file in /usr/include/python3.5 but the compiler can not see that. Since pip install is automatic, it is not easy to include paths for compiling.

tried
pip3 install --global-option=build_ext --global-option="-I/usr/include/python3.5/" --global-option="-L/usr/lib/python3.5" dlib
but didn't work

Current Behavior

Can not attach all of the messages because it is too long

Collecting dlib
Using cached dlib-19.9.0.tar.gz
Building wheels for collected packages: dlib
Running setup.py bdist_wheel for dlib ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__ ='/tmp/pip-build-5gpkn0h7/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f .read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpekkz9n4dpip-wheel- --python-tag cp36:
running bdist_wheel
running build
running build_py
package init file 'dlib/__init__.py' not found (or not a regular file)
running build_ext
Invoking CMake setup: 'cmake /tmp/pip-build-5gpkn0h7/dlib/tools/python -DCMAKE_LIBRARY_OU TPUT_DIRECTORY=/tmp/pip-build-5gpkn0h7/dlib/build/lib.linux-x86_64-3.6 -DPYTHON_EXECUTABLE= /usr/bin/python3 -DCMAKE_BUILD_TYPE=Release'
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3 (found version "3.6.3")
-- Found PythonLibs: python3.6m
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.3.dev0
-- Enabling SSE4 instructions
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux- gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux- gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found X11: /usr/lib/x86_64-linux-gnu/libX11.so
-- Looking for png_create_read_struct
-- Looking for png_create_read_struct - found
-- Looking for jpeg_read_header
-- Looking for jpeg_read_header - found
-- Searching for BLAS and LAPACK
-- Searching for BLAS and LAPACK
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'cblas'
-- No package 'cblas' found
-- Checking for module 'lapack'
-- Found lapack, version 3.10.2
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found LAPACK library
-- Found ATLAS BLAS library
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Looking for sgesv
-- Looking for sgesv - not found
-- Looking for sgesv_
-- Looking for sgesv_ - found
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE _DIRS CUDA_CUDART_LIBRARY) (Required is at least version "7.5")
-- Disabling CUDA support for dlib. DLIB WILL NOT USE CUDA
-- C++11 activated.
-- Found Python with installed numpy package
-- Numpy include path '/usr/local/lib/python3.6/dist-packages/numpy/core/include'
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/pip-build-5gpkn0h7/dlib/build/temp.linux-x86_64 -3.6
Invoking CMake build: 'cmake --build . --config Release -- -j32'
Scanning dependencies of target dlib
[ 2%] Building CXX object dlib_build/CMakeFiles/dlib.dir/entropy_decoder/entropy_decoder _kernel_2.cpp.o
[ 2%] Building CXX object dlib_build/CMakeFiles/dlib.dir/base64/base64_kernel_1.cpp.o
[ 3%] Building CXX object dlib_build/CMakeFiles/dlib.dir/unicode/unicode.cpp.o
[ 5%] Building CXX object dlib_build/CMakeFiles/dlib.dir/data_io/image_dataset_metadata. cpp.o
.
.
.
.
.
.
[ 73%] Building CXX object CMakeFiles/dlib_python.dir/src/dlib.cpp.o
[ 74%] Building CXX object CMakeFiles/dlib_python.dir/src/matrix.cpp.o
[ 75%] Building CXX object CMakeFiles/dlib_python.dir/src/vector.cpp.o
[ 76%] Building CXX object CMakeFiles/dlib_python.dir/src/svm_c_trainer.cpp.o
[ 78%] Building CXX object CMakeFiles/dlib_python.dir/src/decision_functions.cpp.o
[ 79%] Building CXX object CMakeFiles/dlib_python.dir/src/basic.cpp.o
[ 80%] Building CXX object CMakeFiles/dlib_python.dir/src/svm_rank_trainer.cpp.o
[ 82%] Building CXX object CMakeFiles/dlib_python.dir/src/cca.cpp.o
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind1 1/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind1 1/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind1 1/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind1 1/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/dlib.cpp:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:111: 20: fatal error: Python.h: No such file or directory
compilation terminated.

.
.
.
.
.
.
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:230: recipe for target 'CMakeFiles/dlib_python.di r/src/basic.cpp.o' failed
make[2]: * [CMakeFiles/dlib_python.dir/src/basic.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/face_recognition.cp p:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:350: recipe for target 'CMakeFiles/dlib_python.di r/src/rectangles.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/rectangles.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/correlation_tracker .cpp:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:446: recipe for target 'CMakeFiles/dlib_python.di r/src/face_recognition.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/face_recognition.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/gui.cpp:3:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:422: recipe for target 'CMakeFiles/dlib_python.di r/src/correlation_tracker.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/correlation_tracker.cpp.o] Error 1
CMakeFiles/dlib_python.dir/build.make:542: recipe for target 'CMakeFiles/dlib_python.di r/src/gui.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/gui.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/global_optimization .cpp:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/cnn_face_detector.c pp:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/shape_predictor.cpp :4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:494: recipe for target 'CMakeFiles/dlib_python.di r/src/global_optimization.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/global_optimization.cpp.o] Error 1
CMakeFiles/dlib_python.dir/build.make:470: recipe for target 'CMakeFiles/dlib_python.di r/src/cnn_face_detector.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/cnn_face_detector.cpp.o] Error 1
CMakeFiles/dlib_python.dir/build.make:398: recipe for target 'CMakeFiles/dlib_python.di r/src/shape_predictor.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/shape_predictor.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/svm_struct.cpp:4:
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:302: recipe for target 'CMakeFiles/dlib_python.di r/src/svm_struct.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/svm_struct.cpp.o] Error 1
In file included from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pytypes.h:12:0,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/cast.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/attr.h:13,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybin d11/pybind11.h:43,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python/pybind_utils.h:6 ,
from /tmp/pip-build-5gpkn0h7/dlib/dlib/../dlib/python.h:6,
from /tmp/pip-build-5gpkn0h7/dlib/tools/python/src/numpy_returns.cpp:1 :
/tmp/pip-build-5gpkn0h7/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:11 1:20: fatal error: Python.h: No such file or directory
compilation terminated.
CMakeFiles/dlib_python.dir/build.make:518: recipe for target 'CMakeFiles/dlib_python.di r/src/numpy_returns.cpp.o' failed
make[2]: [CMakeFiles/dlib_python.dir/src/numpy_returns.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/dlib_python.dir/all' failed
make[1]: [CMakeFiles/dlib_python.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: [all] Error 2
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-5gpkn0h7/dlib/setup.py", line 238, in
'Topic :: Software Development',
File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 129, in se tup
return distutils.core.setup(*attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
self.run_command('build')
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-build-5gpkn0h7/dlib/setup.py", line 119, in run
self.build_extension(ext)
File "/tmp/pip-build-5gpkn0h7/dlib/setup.py", line 155, in build_extension
subprocess.check_call(cmake_build, cwd=build_folder)
File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release' , '--', '-j32']' returned non-zero exit status 2.

----------------------------------------

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-5gpkn 0h7/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n' , '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-g_c9naqk -record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-5gpkn0h7/dlib/

Steps to Reproduce

pip3 install dlib

• Version:
  dlib-19.9.0.tar.gz

• Where did you get dlib:
  pip3 ubuntu repository

• Platform:
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description: Ubuntu 16.04.3 LTS
  Release: 16.04
  Codename: xenial

• Compiler:
  gcc (Ubuntu 5.4.0-6ubuntu1~16.04.6) 5.4.0 20160609

[davisking]

You don't have python3-dev installed.

[kilho]

I am pretty sure that I have installed python3-dev
when I type

apt-get install python3-dev

I got

Reading package lists... Done
Building dependency tree
Reading state information... Done
python3-dev is already the newest version (3.5.1-3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[kilho]

[Resolved]
I didn't have python3.6-dev although I am using python3.6.3.
If you type
apt-get install python3-dev
it will install python3.5-dev by default, not python3.6-dev

So installing python3.6-dev,
apt-get install python3.6-dev
Compilation successful

Thank Davisking

[theoldgun]

@kilho ，@Davis E. King I encountered the same problem with you, but I still reported the same error after I installed python3.6-dev. How can I solve it?

[kilho]

Main idea of this issue is that you have to install the same version of python-dev as python version that you are using for the compilation. So,

1. check your python version by typing
   python --version
   if it returns, for instance
   Python3.5.3

2. install the same version of python-dev
   apt-get install python3.5-dev

[martynoconnor]

This didn't solve the problem for me. I have 3.6 and 3.6-dev installed, but still get the same error.

[Nannigalaxy]

If error persists, remove pythonX-dev package and try other version suitable. Solved mine by that

[jemerocay]

[Resolved]
I didn't have python3.6-dev although I am using python3.6.3.
If you type
apt-get install python3-dev
it will install python3.5-dev by default, not python3.6-dev

So installing python3.6-dev,
apt-get install python3.6-dev
Compilation successful

Thank Davisking

The same solution applies to python3.7
apt-get install python3.7-dev

[LucienXian]

@theoldgun @martynoconnor Maybe it is related to the memory of your machine. When I set the machine's swap space to 1G, it will be OK!