Theano: Can not use cuDNN on context None: cannot compile with cuDNN.

Created on 21 Jun 2017  ·  25Comments  ·  Source: Theano/Theano

I just updated theano to 0.10 version with pip:
pip install --no-deps git+https://github.com/Theano/Theano.git#egg=Theano

But it cannot be imported successfully, with the error message as follows:
Can not use cuDNN on context None: cannot compile with cuDNN. We got this error: c:\users\yl~1\appdata\local\temp\try_flags_jbgv_m.c:4:19: fatal error: cudnn.h: No such file or directory compilation terminated. Mapped name None to device cuda: GeForce 940M (0000:01:00.0)

I haven't configured theano to use cudnn. Theano should works without it.

My .theanorc is:
[cuda]
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v7.5

[global]
device = cuda
floatX = float32
allow_input_downcast=True

[nvcc]
fastmath = True
compiler_bindir = C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin

Without the .theanorc file, theano works fine.

Most helpful comment

@deepali-c Bybassing the check doesnt mean solving..

All 25 comments

Have U solved it? I have tried to install cudnn at first, but I cant even install cudnn successfully.

@jingweimo I am going to fix this problem theses Days. Maybe we can keep contact with each other per Email. [email protected]

I observed this error with both old and new backends, for the case when cuDNN is not installed. I am not using a theanorc file.

I investigated on it and below is the output on the old backend:

Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano.sandbox.cuda as cuda
WARNING (theano.tensor.blas): Failed to import scipy.linalg.blas, and Theano flag blas.ldflags is empty. Falling back on s       lower implementations for dot(matrix, vector), dot(vector, matrix) and dot(vector, vector) (No module named scipy.linalg.b       las)
>>> print cuda.dnn.dnn_available.msg
None
>>> print cuda.dnn.dnn_available()
False
>>> print cuda.dnn.dnn_available.msg
Can not compile with cuDNN. We got this error:
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release        (Use -Wno-deprecated-gpu-targets to suppress warning).
/tmp/try_flags_zxAO_r.c:5:19: fatal error: cudnn.h: No such file or directory
compilation terminated

.

As a workaround, use dnn.enabled=False to bypass the cuDNN check.

THEANO_FLAGS='device=cuda0,dnn.enabled=False' python -c "import theano";
Can not use cuDNN on context None: Disabled by dnn.enabled flag
Mapped name None to device cuda0: Tesla K80 (0002:03:00.0)

@tswc: I have been looking around to seek solutions, but still haven't made further progress.

@tswc : let's work out the problem

@deepali-c : this is probably the best solution ever

@deepali-c Bybassing the check doesnt mean solving..

The original question in this issue is about a message getting printed ("cannot compile with cuDNN. We got this error: ..."), even though it does not actually crash or prevent the execution of Theano, it just explains why it is not using cuDNN.
Disabling the check (and the printing of that message, replacing it with "Disabled by dnn.enabled flag") is the appropriate solution, which is why I closed this issue.

Now, if you _want_ to use cuDNN, have the header, but it is not detected, that is a different problem. In that case I encourage you to find existing issues mentioning that problem, or solutions in theano-users and on StackOverflow.
If you are convinced the issues is in Theano and not in your configuration, please open a new issue.

@lamblin But I think there maybe some connections between these 2 problems. So I want to first solve the problem of cudnn.

OP explicitly said:

I haven't configured theano to use cudnn. Theano should works without it.

I understand the error message is the same when cuDNN is purposefully un-installed and when it is not detected but should be there. But the solution in both cases are different.

`>>> import theano

theano.test()
Theano version unknown
theano is installed in C:\ProgramData\Anaconda2\lib\site-packages\theano
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in C:\ProgramData\Anaconda2\lib\site-packages\numpy
Python version 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:17:26) [MSC v.1500 64 bit (AMD64)]
nose version 1.3.7
WARNING (theano.gof.compilelock): Overriding existing lock by dead process '10308' (I am process '4220')
Using cuDNN version 5110 on context None
Mapped name None to device cuda: GeForce GTX 750 Ti (0000:01:00.0)
EE.Eforrtl: error (200): program aborting due to control-C event
Image PC Routine Line Source
libifcoremd.dll 00007FFD719743E4 Unknown Unknown Unknown
KERNELBASE.dll 00007FFD94D75674 Unknown Unknown Unknown
KERNEL32.DLL 00007FFD96EF2D92 Unknown Unknown Unknown
ntdll.dll 00007FFD978D9F64 Unknown Unknown Unknown

(C:\ProgramData\Anaconda2) C:\Windows\system32>`
I am not sure if I have solved this problem.

5463

the answer of deltaz0 may be helpful

@jingweimo

@tswc: I find if sticking with theano 0.9 and device = gpu, then everything goes fine and cudnn is enabled. But I still don't know how to deal with theano 0.10 with device=cuda to enable cudnn.

I got another problem now. No error, but the program still uses the CPU,
even though I set devices=cuda0.

2017年6月21日 19:33,"jingweimo" notifications@github.com写道:

@tswc https://github.com/tswc: I find if sticking with theano 0.9 and
device = gpu, then everything goes fine and cudnn is enabled. But I still
don't know how to deal with theano 0.10 with device=gpu to enable cudnn.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Theano/Theano/issues/6063#issuecomment-310150562, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AbZOOBkpW-rghHv36PL3o0x-oSkaIjk5ks5sGVPQgaJpZM4OAPWu
.

@tswc : what is your version? If 0.9, you can keep seting device=gpu. But it should be noted that the home directory user name can not have space for theano 0.9

I have checked. My Theano theano is Version 0.9.0. But when I set
devices=gpu. There will be some error...

2017年6月21日 19:56,"jingweimo" notifications@github.com写道:

@tswc https://github.com/tswc : what is your version? If 0.9, you can
keep seting device=gpu. But it should be noted that the home directory user
name can not have space for theano 0.9


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Theano/Theano/issues/6063#issuecomment-310157225, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AbZOODL7qBGjdW7IwgR4vUYC3aaR3axJks5sGVlQgaJpZM4OAPWu
.

@tswc: That did not happen to me. 'Import theano' only gives a warning message:

WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). 
Please switch to the gpuarray backend. You can get more information about how to switch at this URL: https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: Quadro K4000 (CNMeM is disabled, cuDNN 5005)`

@jingweimo I have just solved it!!! My testprogram is not the newest~

6065

@tswc : You mean your theano works with cudnn enabled? In this issue, I finally turned to setting enabled=false. But it is only a suboptimal solution. I still have trouble dealing with theano 0.10 with cudnn (https://github.com/Theano/Theano/issues/6093). Can you offer some help?

If you just installed CudaNN: Try to restart your pc and check the CUDA_PATH Environment variable.

I use Visual Studio Code.

@jingweimo Have you resolved your issue yet?

I found a solution, in that theano found the cudnn.h; but now it cannot find driver_types.h.

My setup:
Linux 5.3.0-40-generic
Ubuntu 18.04.1 x86_64
theano 1.0.4
cudnn 7.6.5
cudatoolkit 10.2.89

I installed theano, cudnn, and cudatoolkit via conda
conda install theano cudnn cudatoolkit

The cudnn.h was then located in $HOME/anaconda3/envs/<env_name>/include/cudnn.h

If I set the environment variable to
export CUDA_PATH=$HOME/anaconda3/envs/<env_name>/include/cudnn.h

python -c "import theano" finds cudnn.h; but now it cannot find driver_types.h.

Note that driver_types.h cannot be found anywhere on my system, even though it is supposed to be default with NVIDIA drivers. So the same fix will not work for me.

Hi Fraine,

Plz check nvdia driver and toolkit, try cmd to find bin\nvcc:
$nvidia-smi

Update libgpuarray.

or try install via:

conda install -c mila-udem pygpu
Dont forget update g++/gcc.

On Fri, Mar 13, 2020, 03:01 Jonathan Fraine notifications@github.com
wrote:

I found a solution, in that theano stopped found the cudnn.h; but now
it cannot find driver_types.h.

My setup:
Linux 5.3.0-40-generic
Ubuntu 18.04.1 x86_64
theano 1.0.4
cudnn 7.6.5
cudatoolkit 10.2.89

I install theano, cudnn, and cudatoolkit via conda
The cudnn.h was then located in
$HOME/anaconda3/envs//include/cudnn.h

If I set the environment variable to
export CUDA_PATH=$HOME/anaconda3/envs//include/cudnn.h

python -c "import theano" finds cudnn.h; but now it cannot find
driver_types.h.

Note that driver_types.h cannot be found anywhere on my system, even
though it is supposed to be default with NVIDIA drivers. So the same fix
will not work for me.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Theano/Theano/issues/6063#issuecomment-598386941, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AEYAMLY4OCGFKJ2PPIYIJ6TRHE5R7ANCNFSM4DQA6WXA
.

I found a _solution_, in that theano found the cudnn.h; but now it cannot find driver_types.h.

My setup:
Linux 5.3.0-40-generic
Ubuntu 18.04.1 x86_64
theano 1.0.4
cudnn 7.6.5
cudatoolkit 10.2.89

I installed theano, cudnn, and cudatoolkit via conda
conda install theano cudnn cudatoolkit

The cudnn.h was then located in $HOME/anaconda3/envs/<env_name>/include/cudnn.h

If I set the environment variable to
export CUDA_PATH=$HOME/anaconda3/envs/<env_name>/include/cudnn.h

python -c "import theano" finds cudnn.h; but now it cannot find driver_types.h.

Note that driver_types.h cannot be found anywhere on my system, even though it is supposed to be default with NVIDIA drivers. So the same fix will not work for me.

[Sidenote]: even before finding cudnn.h, the gpu_test.py script here did use my GPU; so I think that this means cudatoolkit and the NVIDIA drivers are working just fine.

I have the same issue with you, driver_types.h cannot be found anywhere on my system.
Have you resolved this issue yet?

@BarryRun @exowanderer same thing here.. trying to solve this for a while now.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

kaare-mikkelsen picture kaare-mikkelsen  ·  5Comments

guoxuesong picture guoxuesong  ·  8Comments

junku901 picture junku901  ·  4Comments

ibab picture ibab  ·  4Comments

dnikolayev picture dnikolayev  ·  8Comments