Nvidia-docker: docker: Error response from daemon: Unknown runtime specified nvidia.
I know that this is a common bug, but I have read a variety of issues + bug reports as well as https://github.com/nvidia/nvidia-container-runtime#docker-engine-setup and still cannot resolve the issue.
1. Issue or feature description
$ docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
docker: Error response from daemon: Unknown runtime specified nvidia.
2. Steps to reproduce the issue
I follow the Ubuntu instructions here: https://github.com/NVIDIA/nvidia-docker
I have verified that nvidia-docker is not installed, and nvidia-docker2 is.
My daemon.json is reasonable and set by the package install:
$ cat /etc/docker/daemon.json
{
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}
I try to restart the dockerd as suggested here https://github.com/nvidia/nvidia-container-runtime#docker-engine-setup:
sudo pkill -SIGHUP dockerd
But still get:
$ docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
docker: Error response from daemon: Unknown runtime specified nvidia.
3. Information to attach (optional if deemed irrelevant)
- [ ] Kernel version from
uname -a
Linux Ubuntu-1804-bionic-64-minimal 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux - [ ] Any relevant kernel output lines from
dmesg
n/a - [ ] Driver information from
nvidia-smi -a
==============NVSMI LOG==============
Timestamp : Sat Oct 6 19:32:52 2018
Driver Version : 410.48
Attached GPUs : 1
GPU 00000000:01:00.0
Product Name : GeForce GTX 1080
Product Brand : GeForce
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : Disabled
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : N/A
Pending : N/A
Serial Number : N/A
GPU UUID : GPU-761e2aa7-1578-4bdc-12c8-d7cac834813a
Minor Number : 0
VBIOS Version : 86.04.17.00.01
MultiGPU Board : No
Board ID : 0x100
GPU Part Number : N/A
Inforom Version
Image Version : G001.0000.01.03
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization mode : None
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x01
Device : 0x00
Domain : 0x0000
Device Id : 0x1B8010DE
Bus Id : 00000000:01:00.0
Sub System Id : 0x119E10DE
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : 1000 KB/s
Rx Throughput : 1000 KB/s
Fan Speed : 41 %
Performance State : P0
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 8119 MiB
Used : 0 MiB
Free : 8119 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 2 MiB
Free : 254 MiB
Compute Mode : Default
Utilization
Gpu : 4 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending : N/A
Temperature
GPU Current Temp : 43 C
GPU Shutdown Temp : 99 C
GPU Slowdown Temp : 96 C
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : Supported
Power Draw : 35.90 W
Power Limit : 180.00 W
Default Power Limit : 180.00 W
Enforced Power Limit : 180.00 W
Min Power Limit : 90.00 W
Max Power Limit : 180.00 W
Clocks
Graphics : 974 MHz
SM : 974 MHz
Memory : 5005 MHz
Video : 873 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 1911 MHz
SM : 1911 MHz
Memory : 5005 MHz
Video : 1708 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes : None
- [ ] Docker version from
docker version
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:24:51 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:28:38 2018
OS/Arch: linux/amd64
Experimental: false
- [ ] NVIDIA packages version from
dpkg -l '*nvidia*'_or_rpm -qa '*nvidia*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================================================================-=====================================-=====================================-========================================================================================================================================
un libgldispatch0-nvidia <none> <none> (no description available)
ii libnvidia-cfg1-410:amd64 410.48-0ubuntu1 amd64 NVIDIA binary OpenGL/GLX configuration library
un libnvidia-cfg1-any <none> <none> (no description available)
un libnvidia-common <none> <none> (no description available)
ii libnvidia-common-410 410.48-0ubuntu1 all Shared files used by the NVIDIA libraries
ii libnvidia-compute-410:amd64 410.48-0ubuntu1 amd64 NVIDIA libcompute package
ii libnvidia-container-tools 1.0.0-1 amd64 NVIDIA container runtime library (command-line tools)
ii libnvidia-container1:amd64 1.0.0-1 amd64 NVIDIA container runtime library
un libnvidia-decode <none> <none> (no description available)
ii libnvidia-decode-410:amd64 410.48-0ubuntu1 amd64 NVIDIA Video Decoding runtime libraries
un libnvidia-encode <none> <none> (no description available)
ii libnvidia-encode-410:amd64 410.48-0ubuntu1 amd64 NVENC Video Encoding runtime library
un libnvidia-fbc1 <none> <none> (no description available)
ii libnvidia-fbc1-410:amd64 410.48-0ubuntu1 amd64 NVIDIA OpenGL-based Framebuffer Capture runtime library
un libnvidia-gl <none> <none> (no description available)
ii libnvidia-gl-410:amd64 410.48-0ubuntu1 amd64 NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
un libnvidia-ifr1 <none> <none> (no description available)
ii libnvidia-ifr1-410:amd64 410.48-0ubuntu1 amd64 NVIDIA OpenGL-based Inband Frame Readback runtime library
un nvidia-304 <none> <none> (no description available)
un nvidia-340 <none> <none> (no description available)
un nvidia-384 <none> <none> (no description available)
un nvidia-390 <none> <none> (no description available)
ii nvidia-compute-utils-410 410.48-0ubuntu1 amd64 NVIDIA compute utilities
ii nvidia-container-runtime 2.0.0+docker18.06.1-1 amd64 NVIDIA container runtime
ii nvidia-container-runtime-hook 1.4.0-1 amd64 NVIDIA container runtime hook
ii nvidia-cuda-dev 9.1.85-3ubuntu1 amd64 NVIDIA CUDA development files
ii nvidia-cuda-doc 9.1.85-3ubuntu1 all NVIDIA CUDA and OpenCL documentation
ii nvidia-cuda-gdb 9.1.85-3ubuntu1 amd64 NVIDIA CUDA Debugger (GDB)
ii nvidia-cuda-toolkit 9.1.85-3ubuntu1 amd64 NVIDIA CUDA development toolkit
ii nvidia-dkms-410 410.48-0ubuntu1 amd64 NVIDIA DKMS package
un nvidia-dkms-kernel <none> <none> (no description available)
un nvidia-docker <none> <none> (no description available)
ii nvidia-docker2 2.0.3+docker18.06.1-1 all nvidia-docker CLI wrapper
un nvidia-driver <none> <none> (no description available)
ii nvidia-driver-410 410.48-0ubuntu1 amd64 NVIDIA driver metapackage
un nvidia-driver-binary <none> <none> (no description available)
un nvidia-kernel-common <none> <none> (no description available)
ii nvidia-kernel-common-410 410.48-0ubuntu1 amd64 Shared files used with the kernel module
un nvidia-kernel-source <none> <none> (no description available)
ii nvidia-kernel-source-410 410.48-0ubuntu1 amd64 NVIDIA kernel source package
un nvidia-legacy-340xx-vdpau-driver <none> <none> (no description available)
un nvidia-libopencl1 <none> <none> (no description available)
un nvidia-libopencl1-dev <none> <none> (no description available)
ii nvidia-modprobe 410.48-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files
ii nvidia-opencl-dev:amd64 9.1.85-3ubuntu1 amd64 NVIDIA OpenCL development files
un nvidia-opencl-icd <none> <none> (no description available)
un nvidia-persistenced <none> <none> (no description available)
ii nvidia-prime 0.8.8 all Tools to enable NVIDIA's Prime
ii nvidia-profiler 9.1.85-3ubuntu1 amd64 NVIDIA Profiler for CUDA and OpenCL
ii nvidia-settings 410.48-0ubuntu1 amd64 Tool for configuring the NVIDIA graphics driver
un nvidia-settings-binary <none> <none> (no description available)
un nvidia-smi <none> <none> (no description available)
un nvidia-utils <none> <none> (no description available)
ii nvidia-utils-410 410.48-0ubuntu1 amd64 NVIDIA driver support binaries
un nvidia-vdpau-driver <none> <none> (no description available)
ii nvidia-visual-profiler 9.1.85-3ubuntu1 amd64 NVIDIA Visual Profiler for CUDA and OpenCL
ii xserver-xorg-video-nvidia-410 410.48-0ubuntu1 amd64 NVIDIA binary Xorg driver
- [ ] NVIDIA container library version from
nvidia-container-cli -V
version: 1.0.0
build date: 2018-09-20T20:19+00:00
build revision: 881c88e2e5bb682c9bb14e68bd165cfb64563bb1
build compiler: x86_64-linux-gnu-gcc-7 7.3.0
build platform: x86_64
build flags: -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -DNDEBUG -std=gnu11 -O2 -g -fdata-sections -ffunction-sections -fstack-protector -fno-strict-aliasing -fvisibility=hidden -Wall -Wextra -Wcast-align -Wpointer-arith -Wmissing-prototypes -Wnonnull -Wwrite-strings -Wlogical-op -Wformat=2 -Wmissing-format-attribute -Winit-self -Wshadow -Wstrict-prototypes -Wunreachable-code -Wconversion -Wsign-conversion -Wno-unknown-warning-option -Wno-format-extra-args -Wno-gnu-alignof-expression -Wl,-zrelro -Wl,-znow -Wl,-zdefs -Wl,--gc-sections
[ ] NVIDIA container library logs (see troubleshooting)
n/a[ ] Docker command, image and tag used
See above
turian
👍15
All 19 comments
Seems to have resolved itself
turian
on 7 Oct 2018
😕11
👎8
Could you kindly specify in detail, what did you do to let nvidia-docker work? I also have Ubuntu 18.04, nvidia-docker2 and followed the installation instructions in this repository.
steve@gx501:~/install/docker$ docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
docker: Error response from daemon: Unknown runtime specified nvidia.
See 'docker run --help'.
steve@gx501:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
md5-1399e76df5f858ba9ccece35e15b63c5
steve@gx501:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
md5-1399e76df5f858ba9ccece35e15b63c5
steve@gx501:~/install/docker$ nvidia-container-cli -V
version: 1.0.0
build date: 2018-09-20T20:19+00:00
build revision: 881c88e2e5bb682c9bb14e68bd165cfb64563bb1
build compiler: x86_64-linux-gnu-gcc-7 7.3.0
build platform: x86_64
build flags: -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -DNDEBUG -std=gnu11 -O2 -g -fdata-sections -ffunction-sections -fstack-protector -fno-strict-aliasing -fvisibility=hidden -Wall -Wextra -Wcast-align -Wpointer-arith -Wmissing-prototypes -Wnonnull -Wwrite-strings -Wlogical-op -Wformat=2 -Wmissing-format-attribute -Winit-self -Wshadow -Wstrict-prototypes -Wunreachable-code -Wconversion -Wsign-conversion -Wno-unknown-warning-option -Wno-format-extra-args -Wno-gnu-alignof-expression -Wl,-zrelro -Wl,-znow -Wl,-zdefs -Wl,--gc-sections
mmaslennikov
on 2 Nov 2018
I've also installed correctly, but forgot to restart daemon in ubuntu. it may resolve your error.
sudo systemctl daemon-reload
sudo systemctl restart docker
sngjuk
on 16 Nov 2018
👍169
❤39
🎉35
👎22
🚀18
👀17
😄12
I have same issue, but the advise of @sngjuk works for me. Thank you!!!
terasakisatoshi
on 17 Nov 2018
I have the same issue, the solution of @sngjuk works for me too but I have to restart docker (sudo systemctl restart docker) each time I reboot the system, it's quite annoying. Does someone has an idea why that is and how to make it work at startup?
francoisdelarbre
on 2 Apr 2019
Works for me too. Thanks for @sngjuk
shadowwalker00
on 28 Jun 2019
@francoisdelarbre Did you ever check your system have the only one of docker service? I have fixed the same issue by disabling the snap.docker.dockerd.service, it is found by the command systemctl list-unit-files | grep enable | grep docker
samwuTW
on 8 Jul 2019
🎉4
👍2
@samwuTW No, I didn't. You are right, I had several docker services. I disabled snap.docker.dockerd.service (systemctl disable snap.docker.dockerd.service) as you did and it works! Thanks for your help!
francoisdelarbre
on 8 Jul 2019
👍1
$ docker run --runtime=nvidia -it nvcr.io/nvidia/tlt-streamanalytics:v1.0_py2 /bin/bash
docker: Error response from daemon: Unknown runtime specified nvidia.
See 'docker run --help'.
Got the above error. Any idea to solve?
Samjith888
on 27 Nov 2019
maybe --gpus all instead of --runtime=nvidia.
terasakisatoshi
on 27 Nov 2019
👍112
🎉25
❤22
🚀9
😕5
👀2
I have solved the error by using following command.
sudo pkill -SIGHUP dockerd
Samjith888
on 27 Nov 2019
👍1
@terasakisatoshi works for me ! Thank you! but why?
pengfeidip
on 9 Jan 2020
@terasakisatoshi solution worked for me too. Thanks!
mzahana
on 28 Feb 2020
I am encountering this for the first time after using snap for installing Docker. I think I'll avoid using snap for Docker going forward.
AVSuni
on 15 Mar 2020
sudo pkill -SIGHUP dockerd, this solved my problem too
fatehOurghi
on 20 Apr 2020
I fixed this with:
sudo apt install -y nvidia-docker2
sudo systemctl daemon-reload
sudo systemctl restart docker
cdpath
on 1 Jun 2020
👍19
🚀2
❤2
🎉2
Thank you this helped for me
Darshcg
on 15 Jun 2020
@cdpath
$ sudo apt install -y nvidia-docker2
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-docker2
monacv
on 7 Jan 2021
@cdpath
$ sudo apt install -y nvidia-docker2 Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package nvidia-docker2
Setting the GPG and remote repo for the package:
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
Save that file and refresh the package list
$ sudo apt-get update
Install nvidia-docker2 and reload the Docker configurations
$ sudo apt-get install -y nvidia-docker2
$ sudo pkill -SIGHUP dockerd
ifarshgar
on 27 Jan 2021
Was this page helpful?
0 / 5 - 0 ratings
Related issues
henry-blip
路
3Comments
djglowny
路
3Comments
vak
路
4Comments
romansavrulin
路
4Comments
adbeda
路
3Comments
Most helpful comment
I've also installed correctly, but forgot to restart daemon in ubuntu. it may resolve your error.