Fairseq: _temporal_convolution_tbc.c:434:21: fatal error: THC/THC.h: No such file or directory

Created on 25 Sep 2017 · 4Comments · Source: pytorch/fairseq

Hi, when I'm trying to install following your instruction, I got that error.
During: python setup.py build

This is what I got:

running install
running bdist_egg
running egg_info
writing fairseq.egg-info/PKG-INFO
writing dependency_links to fairseq.egg-info/dependency_links.txt
writing requirements to fairseq.egg-info/requires.txt
writing top-level names to fairseq.egg-info/top_level.txt
reading manifest file 'fairseq.egg-info/SOURCES.txt'
writing manifest file 'fairseq.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
generating /tmp/tmpj5grradm/_temporal_convolution_tbc.c
running build_ext
building '_temporal_convolution_tbc' extension
creating nfs
creating nfs/guille
creating nfs/guille/huang
creating nfs/guille/huang/users
creating nfs/guille/huang/users/yangyil
creating nfs/guille/huang/users/yangyil/fairseq-py
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq/clib
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq/clib/temporal_convolution_tbc
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/nfs/stak/users/yangyil/shared/miniconda3/include/python3.6m -c _temporal_convolution_tbc.c -o ./_temporal_convolution_tbc.o -std=c++11
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C [enabled by default]
_temporal_convolution_tbc.c:434:21: fatal error: THC/THC.h: No such file or directory
 #include <THC/THC.h>
                     ^
compilation terminated.
Traceback (most recent call last):
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 49, in _build
    dist.run_command('build_ext')
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 198, in build_extension
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 69, in <module>
    'build_py': build_py_hook,
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/install.py", line 67, in run
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/install.py", line 109, in do_egg_install
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/bdist_egg.py", line 161, in run
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/bdist_egg.py", line 147, in call_command
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/install_lib.py", line 11, in run
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/install_lib.py", line 105, in build
    self.run_command('build_py')
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 49, in run
    conv_tbc.build()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 164, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 100, in _build_extension
    ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/api.py", line 684, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/recompiler.py", line 1484, in recompile
    compiler_verbose, debug)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 20, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 56, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'gcc' failed with exit status 1

I build pytorch from the source, and reset it as you indicate.
This is all the library I installed in this environment.

$ conda list
# packages in environment at /nfs/guille/huang/users/yangyil/miniconda3:
#
asn1crypto                0.22.0                   py36_0
cffi                      1.10.0                   py36_0
cloog                     0.18.0                        0
cmake                     0.8.0                     <pip>
conda                     4.3.25                   py36_0
conda-env                 2.6.0                         0
cryptography              1.8.1                    py36_0
cuda80                    1.0                           0    soumith
cudatoolkit               8.0                           1
cudnn                     6.0.21                cuda8.0_0
gcc                       4.8.5                         7
gmp                       6.1.0                         0
idna                      2.5                      py36_0
isl                       0.12.2                        0
libffi                    3.2.1                         1
magma-cuda80              2.1.0                         5    soumith
mkl                       2017.0.3                      0
mpc                       1.0.3                         0
mpfr                      3.1.5                         0
nccl                      1.3.4                 cuda8.0_1
numpy                     1.13.1                   py36_0
openssl                   1.0.2l                        0
packaging                 16.8                     py36_0
pip                       9.0.1                    py36_1
pycosat                   0.6.2                    py36_0
pycparser                 2.17                     py36_0
pyopenssl                 17.0.0                   py36_0
pyparsing                 2.1.4                    py36_0
python                    3.6.1                         2
PyYAML                    3.12                      <pip>
readline                  6.2                           2
requests                  2.14.2                   py36_0
ruamel_yaml               0.11.14                  py36_1
setuptools                27.2.0                   py36_0
six                       1.10.0                   py36_0
sqlite                    3.13.0                        0
tk                        8.5.18                        0
torch                     0.2.0+a03e5cb             <pip>
tqdm                      4.17.1                    <pip>
wheel                     0.29.0                   py36_0
xz                        5.2.2                         1
yaml                      0.1.6                         0
zlib                      1.2.8                         3

Source

yilinyang7

Most helpful comment

@andyang the problem is that it can't find cuda.h in your build path. Can you try adding it manually, e.g. by running this before building fairseq?

export CPATH=/usr/local/apps/cuda/cuda-8.0/include

I'm assuming the path for headers in your system is /usr/local/apps/cuda/cuda-8.0/include but you can also check if it contains cuda.h, if not try to find where it is in your system

edunov on 27 Sep 2017

👍8 🎉4

All 4 comments

@andyang it seems that pytorch couldn't detect CUDA during the installation. Can you please check if you have CUDA installed in your system. E.g. by running nvcc --version. You should see something like this:

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

And then, once you have cuda sorted out try to clean-install pytorch again.
If it doesn't help, please attach pytorch build log to the issue.

edunov on 25 Sep 2017

Thanks, you were right, I didn't have CUDA lib in my $PATH before.
After adding to my path, I remove & reclone & reinstall the pytorch.
but now, there's a different error..

$ python setup.py build
running build
running build_py
generating /tmp/tmpfy0uiu9a/_temporal_convolution_tbc.c
setting the current directory to '/tmp/tmpfy0uiu9a'
running build_ext
building '_temporal_convolution_tbc' extension
creating nfs
creating nfs/guille
creating nfs/guille/huang
creating nfs/guille/huang/users
creating nfs/guille/huang/users/yangyil
creating nfs/guille/huang/users/yangyil/fairseq-py
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq/clib
creating nfs/guille/huang/users/yangyil/fairseq-py/fairseq/clib/temporal_convolution_tbc
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/nfs/stak/users/yangyil/shared/miniconda3/include/python3.6m -c _temporal_convolution_tbc.c -o ./_temporal_convolution_tbc.o -std=c++11
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C [enabled by default]
In file included from /nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC/THC.h:4:0,
                 from _temporal_convolution_tbc.c:483:
/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC/THCGeneral.h:9:18: fatal error: cuda.h: No such file or directory
 #include "cuda.h"
                  ^
compilation terminated.
Traceback (most recent call last):
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 198, in build_extension
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 69, in <module>
    'build_py': build_py_hook,
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 49, in run
    conv_tbc.build()
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 164, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 100, in _build_extension
    ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/api.py", line 690, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/recompiler.py", line 1513, in recompile
    compiler_verbose, debug)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/nfs/stak/users/yangyil/shared/miniconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'gcc' failed with exit status 1

In addition:

$ echo $LD_LIBRARY_PATH
/usr/local/apps/cuda/cuda-8.0/lib64:

$ ls /usr/local/apps/cuda/cuda-8.0/lib64/libcudnn*
/usr/local/apps/cuda/cuda-8.0/lib64/libcudnn.so
/usr/local/apps/cuda/cuda-8.0/lib64/libcudnn.so.5
/usr/local/apps/cuda/cuda-8.0/lib64/libcudnn.so.5.1.5
/usr/local/apps/cuda/cuda-8.0/lib64/libcudnn_static.a

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44

$ ipython
Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import torch

In [2]: torch.cuda.is_available()
Out[2]: True

yilinyang7 on 26 Sep 2017

@andyang the problem is that it can't find cuda.h in your build path. Can you try adding it manually, e.g. by running this before building fairseq?

export CPATH=/usr/local/apps/cuda/cuda-8.0/include

I'm assuming the path for headers in your system is /usr/local/apps/cuda/cuda-8.0/include but you can also check if it contains cuda.h, if not try to find where it is in your system

edunov on 27 Sep 2017

👍8 🎉4

Thanks so much.
It's actually a very newbie question.

yilinyang7 on 27 Sep 2017

Was this page helpful?

0 / 5 - 0 ratings