Nvidia-docker: Why can't I run command “nvcc --version" in docker-CUDA Container?

[opptimus]

My OS is Ubuntu 18.04，and already install nvidia-docker.
When I run

docker run -it --gpus all nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04 /bin/bash
root@7c0be9bfaeec:/# nvcc --version
bash: nvcc: command not found

Also Cannot detect CUDNN at the same time in the docker Container when I use:

root@694cefcad33a:/# cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
cat: /usr/local/cuda/include/cudnn.h: No such file or directory

However,run nvidia-smi ,it worked!
root@7c0be9bfaeec:/# nvidia-smi
Thu Dec 19 04:51:23 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.44 Driver Version: 440.44 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |

[dualvtable]

Hi there,

To get access to the CUDA development tools, you should use the devel images instead. These are the relevant tags:

1. nvidia/cuda:10.2-devel-ubuntu18.04
2. nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04

These would then give you access to the tools such as nvcc and the cuDNN header files that are required for development.

Hope this helps!

[opptimus]

Thank you very much, the problem has been solved

[mirekphd]

Note that this requirement for devel nvidia/cuda images (which are incidentally larger by 2 GiB than runtime ones) applies to CUDA 11 as well. You can encounter this issue of missing NVIDIA CUDA compiler nvcc e.g. when attempting to compile xgboost for GPU (with -DUSE_CUDA=ON) in a smaller runtime image:
https://github.com/microsoft/LightGBM/issues/3040#issuecomment-669851228