Theano: cannot compile with cuDNN

Created on 14 May 2017  路  9Comments  路  Source: Theano/Theano

Hello,

Getting an error running Theano (bleeding-edge version) with cuda 8.0, cuDNN 6.0 on Linux (Ubuntu 16.04)

$ python -c "import theano"
Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
In file included from /tmp/try_flags_Y9EarB.c:4:0:
/usr/include/cudnn.h:63:26: fatal error: driver_types.h: No such file or directory
#include "driver_types.h"
^
compilation terminated.

Mapped name None to device cuda0: GeForce GTX 1060 6GB (0000:01:00.0)

Have tried changing to #include and theano-cache purge. I've reinstalled cuDNN and placed the cudnn.h file in the cuda-8.0 folder and added paths in the theanorc, then tried moving the cudnn.h file from /usr/include/ now getting this error.

$ python -c "import theano"
Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
/tmp/try_flags_7l4zMr.c:4:19: fatal error: cudnn.h: No such file or directory
#include
^
compilation terminated.

Mapped name None to device cuda0: GeForce GTX 1060 6GB (0000:01:00.0)

Here is the theanorc:

[global]
floatX = float32
device = cuda0

[lib]
cnmem = 1

[nvcc]
flags=-D_FORCE_INLINES
fastmath = True

[blas]
ldflags = -lopenblas

[cuda]
root = /usr/local/cuda-8.0

[dnn]
library_path = /usr/local/cuda-8.0/lib64
include_path = /usr/local/cuda-8.0/include

Most helpful comment

in case you passed the cudnn test [https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#verify], yet still facing the same problem for Theano:

  1. in ~/.theanorc, ensure you have:
    [cuda]
    root=/usr/local/cuda/bin
    [dnn]
    include_path=/usr/local/cuda/include
    library_path=/usr/local/cuda/lib64
  2. cd /usr/local/cuda/include
    sudo cp /usr/include/cudnn.h .
    cd /usr/local/cuda/lib64
    sudo cp /usr/lib/x86_64-linux-gnu/libcudnn* .

  3. python -c 'import theano'
    Using cuDNN version 7103 on context None
    Mapped name None to device cuda: GeForce GTX 1080 Ti (0000:02:00.0)

passed.

All 9 comments

Seems like you didn't install cudnn correctly.

Thanks. Have found the I can still run projects, the warning just comes up each time. Found this Issue which looks pretty similar, and will try and resolve with the methods suggested when I have more time. https://github.com/Theano/Theano/issues/5463

[cuda]
root = /usr/local/cuda-8.0

should be:

[cuda]
root = /usr/local/cuda-8.0/bin

Thank you. Paths to cudNN were incorrect. Bashrc and theanorc have been updated. Now getting this RuntimeError when trying to run a neural-network... any ideas?

RuntimeError: ('The following error happened while compiling the node', GpuAdvancedIncSubtensor1_dev20{inplace=True, set_instead_of_inc=False}(GpuAlloc{memset_0=True}.0, GpuReshape{2}.0, GpuReshape{1}.0), 'n', 'GpuKernel_init error 3: nvrtcCompileProgram: NVRTC_ERROR_COMPILATION')


$ python -c "import theano"
Using cuDNN version 6021 on context None
Mapped name None to device cuda0: GeForce GTX 1060 6GB (0000:01:00.0)

Theanorc:

[global]
floatX = float32
device = cuda0

[lib]
cnmem = 1

[nvcc]
flags=-D_FORCE_INLINES
fastmath = True

[blas]
ldflags = -lopenblas

[cuda]
root = /usr/local/cuda-8.0/bin

[dnn]
include_path = /usr/include/
library_path = /usr/lib/x86_64-linux-gnu

If you recompile libgpuarray in Debug mode and try to run the code again, it should give you more information on what the compilation error is, and the code it is trying to compile.

Hi guys,

I was with the same problem. So, I go to CudNN documentation and my GPU is not compatible.

My GPU is: GTX 560 ti

"A GPU of compute capability 3.0 or higher. To understand the compute capability of the GPU on your system, see: CUDA GPUs."

documentation: http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html

Open a new issue with the full error message.

in case you passed the cudnn test [https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#verify], yet still facing the same problem for Theano:

  1. in ~/.theanorc, ensure you have:
    [cuda]
    root=/usr/local/cuda/bin
    [dnn]
    include_path=/usr/local/cuda/include
    library_path=/usr/local/cuda/lib64
  2. cd /usr/local/cuda/include
    sudo cp /usr/include/cudnn.h .
    cd /usr/local/cuda/lib64
    sudo cp /usr/lib/x86_64-linux-gnu/libcudnn* .

  3. python -c 'import theano'
    Using cuDNN version 7103 on context None
    Mapped name None to device cuda: GeForce GTX 1080 Ti (0000:02:00.0)

passed.

I am having the same problem. I made the recommended changes to my .theanorc, and that did not solve it. I could not do many of the other suggestions because I do not have sudo access. theano still runs for me in spite of this error. I am using theano version 1.0.4. Please let me know if you have suggestions for what I should do. Thanks!

Was this page helpful?
0 / 5 - 0 ratings