Pytorch: [build/nccl] failed to build libnccl on Debian unstable

[cdluminate]

Failed to build CUDA version of pytorch (without CUDNN) with the latest source.

OS: debian unstable/experimental
Compiler: gcc-5, g++-5
CUDA: 8.0.44 (package provided by Debian)

buildlog: http://debomatic-amd64.debian.net/distribution#experimental/pytorch-contrib/0.1.7~1/buildlog

-- The C compiler identification is GNU 5.4.1
-- The CXX compiler identification is GNU 5.4.1
-- Check for working C compiler: /usr/bin/gcc-5
-- Check for working C compiler: /usr/bin/gcc-5 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-5
-- Check for working CXX compiler: /usr/bin/g++-5 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found CUDA: /usr (found suitable version "8.0", minimum required is "7.0") 
-- Configuring done
-- Generating done
-- Build files have been written to: /<<PKGBUILDDIR>>/torch/lib/build/nccl
make[2]: Entering directory '/<<PKGBUILDDIR>>/torch/lib/build/nccl'
make[3]: Entering directory '/<<PKGBUILDDIR>>/torch/lib/build/nccl'
make[4]: Entering directory '/<<PKGBUILDDIR>>/torch/lib/build/nccl'
Scanning dependencies of target nccl
make[4]: Leaving directory '/<<PKGBUILDDIR>>/torch/lib/build/nccl'
make[4]: Entering directory '/<<PKGBUILDDIR>>/torch/lib/build/nccl'
[100%] Generating lib/libnccl.so
make[5]: Entering directory '/<<PKGBUILDDIR>>/torch/lib/nccl'
ls: cannot access '/usr/lib64/libcudart.so.*': No such file or directory
ls: cannot access '/usr/lib64/libcudart.so.*': No such file or directory
Grabbing  src/nccl.h                > /<<PKGBUILDDIR>>/torch/lib/build/nccl/include/nccl.h
Compiling src/libwrap.cu            > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/libwrap.o
Compiling src/core.cu               > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/core.o
Compiling src/all_gather.cu         > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/all_gather.o
Compiling src/all_reduce.cu         > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/all_reduce.o
Compiling src/broadcast.cu          > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/broadcast.o
Compiling src/reduce.cu             > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/reduce.o
Compiling src/reduce_scatter.cu     > /<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/reduce_scatter.o
src/core.cu(724): error: expected an expression

src/core.cu(724): error: expected an expression

2 errors detected in the compilation of "/tmp/tmpxft_00002c02_00000000-13_core.compute_52.cpp1.ii".
Makefile:98: recipe for target '/<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/core.o' failed
make[5]: *** [/<<PKGBUILDDIR>>/torch/lib/build/nccl/obj/core.o] Error 2

I have no idea about this ...

[apaszke]

If you look here you'll see that it fails, because ls can't find libcudart.so, and that's why CUDA_MAJOR and CUDA_MINOR end up being empty strings. I don't know how a debian CUDA package looks, but you probably don't have libcudart.so with a version extension.

[apaszke]

Maybe CUDA_VERSION=8.0 pp setup.py build helps?

[cdluminate]

@apaszke Thanks, the fix is to to export the two environment variables:

export CUDA_HOME=/usr
export CUDA_LIB=/usr/lib/$(shell dpkg-architecture -qDEB_HOST_MULTIARCH)