dlib 19.20 python setup.py install/build failed

[tapas]

Expected Behavior

I was trying to build dlib through: python setup.py install but it failed after 2% build

Current Behavior

Terminal output below:
$ python setup.py install
running build
running build_py
package init file 'dlib/__init__.py' not found (or not a regular file)
running build_ext
Building extension for Python 3.5.2 (default, Nov 23 2017, 16:37:01)
Invoking CMake setup: 'cmake /home/surveillance6/Downloads/dlib-19.20/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/surveillance6/Downloads/dlib-19.20/build/lib.linux-x86_64-3.5 -DPYTHON_EXECUTABLE=/home/surveillance6/virt_envs/mlenv/bin/python -DCMAKE_BUILD_TYPE=Release'
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /home/surveillance6/virt_envs/mlenv/bin/python (found version "3.5.2")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.2.2
-- Using CMake version: 3.5.1
-- Compiling dlib version: 19.20.0
-- SSE4 instructions can be executed by the host processor.
-- AVX instructions can be executed by the host processor.
-- Enabling AVX instructions
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found X11: /usr/lib/x86_64-linux-gnu/libX11.so
-- Found system copy of libpng: /usr/lib/x86_64-linux-gnu/libpng.so;/usr/lib/x86_64-linux-gnu/libz.so
-- Found system copy of libjpeg: /usr/lib/x86_64-linux-gnu/libjpeg.so
-- Searching for BLAS and LAPACK
-- Searching for BLAS and LAPACK
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'cblas'
-- No package 'cblas' found
-- Checking for module 'lapack'
-- Found lapack, version 0.2.18
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found OpenBLAS library
-- Looking for sgetrf_single
-- Looking for sgetrf_single - found
-- Using OpenBLAS's built in LAPACK
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Looking for sgesv
-- Looking for sgesv - not found
-- Looking for sgesv_
-- Looking for sgesv_ - not found
-- Found CUDA: /usr/local/cuda-10.1 (found suitable version "10.1", minimum required is "7.5")
-- Looking for cuDNN install...
-- Found cuDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so
-- Building a CUDA test project to see if your compiler is compatible with CUDA...
-- Checking if you have the right version of cuDNN installed.
-- Enabling CUDA support for dlib. DLIB WILL USE CUDA
-- C++11 activated.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/surveillance6/Downloads/dlib-19.20/build/temp.linux-x86_64-3.5
Invoking CMake build: 'cmake --build . --config Release -- -j8'
[ 2%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o
[ 2%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cusolver_dlibapi.cu.o
/home/surveillance6/Downloads/dlib-19.20/dlib/cuda/cuda_dlib.cu(1762): error: calling a constexpr __host__ function("log1p") from a __device__ function("cuda_log1pexp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

1 error detected in the compilation of "/tmp/tmpxft_00000261_00000000-6_cuda_dlib.cpp1.ii".
CMake Error at dlib_generated_cuda_dlib.cu.o.cmake:266 (message):
Error generating file
/home/surveillance6/Downloads/dlib-19.20/build/temp.linux-x86_64-3.5/dlib_build/CMakeFiles/dlib.dir/cuda/./dlib_generated_cuda_dlib.cu.o

dlib_build/CMakeFiles/dlib.dir/build.make:63: recipe for target 'dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o' failed
make[2]: * [dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o] Error 1
make[2]: Waiting for unfinished jobs....
CMakeFiles/Makefile2:140: recipe for target 'dlib_build/CMakeFiles/dlib.dir/all' failed
make[1]: [dlib_build/CMakeFiles/dlib.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: [all] Error 2
Traceback (most recent call last):
File "setup.py", line 261, in
'Topic :: Software Development',
File "/home/surveillance6/virt_envs/mlenv/lib/python3.5/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(*attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.5/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 135, in run
self.build_extension(ext)
File "setup.py", line 175, in build_extension
subprocess.check_call(cmake_build, cwd=build_folder)
File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2

Steps to Reproduce

I am using: Ubuntu 16.04, cuda 10.1 cudnn also seems to be installed. was doing python setup.py build inside virtual environment
I am able to successfully build dlib 19.19 in same environment.

• Version: 19.20
• Where did you get dlib: downloaded dlib from dlib.net

• Platform:
  $ lsb_release -a
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description: Ubuntu 16.04.6 LTS
  Release: 16.04
  Codename: xenial

• Compiler:
  $ g++ --version
  g++ (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
  Copyright (C) 2015 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[davisking]

I suspect you have a stale build folder. Get a fresh copy of dlib, make sure you put it in a separate folder (or delete the one you have now) and try installing again.

[gabm]

yep same here with cuda 10.1 and gcc 7.5.0. 19.19 builds fine, 19.20 fails with the above error

[davisking]

Did you do this from a fresh download of dlib?

[gabm]

yes, in a clean build folder...

[davisking]

Let me say that another way, please provide the exact commands to reproduce this, starting with the command that downloads dlib from the internet.

[gabm]

$ git clone https://github.com/davisking/dlib.git
$ cd dlib
$ git checkout v19.20
$ mkdir build && cd build
$ cmake ../tools/python
$ make -j4
make[1]: Entering directory '/home/gabm/dev/os/dlib/build'
make[2]: Entering directory '/home/gabm/dev/os/dlib/build'
[  1%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o
[  1%] Building NVCC (Device) object dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cusolver_dlibapi.cu.o
/home/gabm/dev/os/dlib/dlib/cuda/cuda_dlib.cu(1762): error: calling a constexpr __host__ function("log1p") from a __device__ function("cuda_log1pexp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

1 error detected in the compilation of "/tmp/tmpxft_00016691_00000000-6_cuda_dlib.cpp1.ii".
CMake Error at dlib_generated_cuda_dlib.cu.o.Release.cmake:280 (message):
  Error generating file
  /home/gabm/dev/os/dlib/build/dlib_build/CMakeFiles/dlib.dir/cuda/./dlib_generated_cuda_dlib.cu.o


make[2]: *** [dlib_build/CMakeFiles/dlib.dir/build.make:82: dlib_build/CMakeFiles/dlib.dir/cuda/dlib_generated_cuda_dlib.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/home/gabm/dev/os/dlib/build'
make[1]: *** [CMakeFiles/Makefile2:160: dlib_build/CMakeFiles/dlib.dir/all] Error 2
make[1]: Leaving directory '/home/gabm/dev/os/dlib/build'
make: *** [Makefile:101: all] Error 2

[gabm]

cuda 10.1.243 and cudnn 7.6.5

[davisking]

Running git clone https://github.com/davisking/dlib.git followed immediately by git checkout v19.20 is wrong. You need to at least cd dlib in between. Please post exactly the commands you use. I will run those commands to see if I can reproduce the error.

[gabm]

sorry, thats the only thing that was missing.. I copied the commands along while executing it in the terminal...

[davisking]

Urg, I see why this is happening. The latest update for cuda 10.1 has this bug too. I fixed this in dlib's cmake scripts a while ago by setting --expt-relaxed-constexpr, but only applied that flag for cuda 10.2 (the version where the bug appeared). Since then the bug in cuda has been pushed into cuda 10.1 too, apparently.

Anyway, pull the latest dlib. I just pushed a fix.

[gabm]

yep, that works... thanks for the ultra-fast fix :+1:

[davisking]

No problem

[tapas]

Sorry for late response, to recall I created this thread, as I was getting error with cuda 10.1, I recompiled dlib with 10.2 and it worked but I took time to respond to this thread, and not it seems you have fixed the patch for 10.1 also. Thanks for your time

[mehulhirpara]

I'm facing this issue on Nvidia TX2 with cuda 9.0

[takuya-takeuchi]

This issue looks like happening on cuda 9.2 and 10.0.
it worked fine on 10.1 and 10.2 due to author fixed CMakeLists.txt.
I guess the following condition should contains 9.2 and 10.0.

if (CUDA_VERSION VERSION_EQUAL 10.2 OR CUDA_VERSION VERSION_EQUAL 10.1) # To avoid this error from nvcc: # error: calling a constexpr __host__ function("log1p") from a __device__ function("cuda_log1pexp") # is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this. # which is a bug in nvcc in this version of cuda. list(APPEND FLAGS_FOR_NVCC "--expt-relaxed-constexpr") endif()

After fixed this, it works fine on 9.2 and 10.0.

[davisking]

Yeah, seems to be active there as well. I just changed the cmake scripts to apply to all older versions of cuda instead of just 10.1 and 10.2.

[humandotlearning]

pip install of dlib is also failing with the same error on a ubuntu16.04 docker with:

1. nvidia-driver Version: 440.33.01
2. cuda Version: V10.0.130

@davisking Is their any way to give flag '--expt-relaxed-constexpr' for the python pip install of dlib?

[davisking]

I don't know if there is a way with pip, probably not. I'll update the copy of dlib in pip this weekend though. I assume the copy of dlib on github is working fine for you?