Nvidia-docker: Failed to specify all capability

Created on 15 Nov 2019 · 1Comment · Source: NVIDIA/nvidia-docker

1. Issue or feature description

Failed when specifying all or multiple capabilities for an image:

$ docker run --gpus all,capabilities=all --rm -it ubuntu:trusty
docker: Error response from daemon: could not select device driver "" with capabilities: [[all gpu]].

$ docker run --gpus all,capabilities=compute,utility --rm -it ubuntu:trusty
invalid argument "all,capabilities=compute,utility" for "--gpus" flag: count must be an integer: strconv.Atoi: parsing "utility": invalid syntax
See 'docker run --help'.

2. Steps to reproduce the issue

All of these worked:

$ docker run --gpus all --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=compute --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=compat32 --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=graphics --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=utility --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=video --rm -it ubuntu:trusty
$ docker run --gpus all,capabilities=display --rm -it ubuntu:trusty

Other information:

I've restarted dockerd.
Wiki says to use capabilities=utilities but it seems that only capabilities=utility works.
At the end of nvidia-container information, driver service terminated successfully. Is it a normal case?

3. Information to attach (optional if deemed irrelevant)

[x] Some nvidia-container information: nvidia-container-cli -k -d /dev/tty info

-- WARNING, the following logs are for debugging purposes only --

I1115 05:19:38.321602 13527 nvc.c:281] initializing library context (version=1.0.5, build=13b836390888f7b7c7dca115d16d7e28ab15a836)
I1115 05:19:38.321622 13527 nvc.c:255] using root /
I1115 05:19:38.321646 13527 nvc.c:256] using ldcache /etc/ld.so.cache
I1115 05:19:38.321648 13527 nvc.c:257] using unprivileged user 1000:1000
W1115 05:19:38.322174 13528 nvc.c:186] failed to set inheritable capabilities
W1115 05:19:38.322191 13528 nvc.c:187] skipping kernel modules load due to failure
I1115 05:19:38.322344 13529 driver.c:133] starting driver service
I1115 05:19:38.365907 13527 nvc_info.c:437] requesting driver information with ''
I1115 05:19:38.366136 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvoptix.so.435.21
I1115 05:19:38.366167 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.435.21
I1115 05:19:38.366186 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.435.21
I1115 05:19:38.366217 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.435.21
I1115 05:19:38.366257 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.435.21
I1115 05:19:38.366310 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.435.21
I1115 05:19:38.366340 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.435.21
I1115 05:19:38.366379 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-ifr.so.435.21
I1115 05:19:38.366416 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.435.21
I1115 05:19:38.366448 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.435.21
I1115 05:19:38.366493 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.435.21
I1115 05:19:38.366544 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.435.21
I1115 05:19:38.366567 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.435.21
I1115 05:19:38.366583 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.435.21
I1115 05:19:38.366605 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.435.21
I1115 05:19:38.366623 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-compiler.so.435.21
I1115 05:19:38.366639 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.435.21
I1115 05:19:38.366662 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libnvcuvid.so.435.21
I1115 05:19:38.366804 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libcuda.so.435.21
I1115 05:19:38.366893 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.435.21
I1115 05:19:38.366911 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.435.21
I1115 05:19:38.366928 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.435.21
I1115 05:19:38.366947 13527 nvc_info.c:151] selecting /usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.435.21
I1115 05:19:38.366975 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-tls.so.435.21
I1115 05:19:38.366994 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-ptxjitcompiler.so.435.21
I1115 05:19:38.367020 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-opticalflow.so.435.21
I1115 05:19:38.367045 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-opencl.so.435.21
I1115 05:19:38.367064 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-ml.so.435.21
I1115 05:19:38.367089 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-ifr.so.435.21
I1115 05:19:38.367113 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-glvkspirv.so.435.21
I1115 05:19:38.367130 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-glsi.so.435.21
I1115 05:19:38.367147 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-glcore.so.435.21
I1115 05:19:38.367165 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-fbc.so.435.21
I1115 05:19:38.367190 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-fatbinaryloader.so.435.21
I1115 05:19:38.367207 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-encode.so.435.21
I1115 05:19:38.367231 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-eglcore.so.435.21
I1115 05:19:38.367249 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvidia-compiler.so.435.21
I1115 05:19:38.367268 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libnvcuvid.so.435.21
I1115 05:19:38.367304 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libcuda.so.435.21
I1115 05:19:38.367334 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libGLX_nvidia.so.435.21
I1115 05:19:38.367353 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libGLESv2_nvidia.so.435.21
I1115 05:19:38.367370 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libGLESv1_CM_nvidia.so.435.21
I1115 05:19:38.367387 13527 nvc_info.c:151] selecting /usr/lib/i386-linux-gnu/libEGL_nvidia.so.435.21
W1115 05:19:38.367399 13527 nvc_info.c:302] missing library libvdpau_nvidia.so
W1115 05:19:38.367403 13527 nvc_info.c:306] missing compat32 library libnvidia-cfg.so
W1115 05:19:38.367407 13527 nvc_info.c:306] missing compat32 library libvdpau_nvidia.so
W1115 05:19:38.367410 13527 nvc_info.c:306] missing compat32 library libnvidia-rtcore.so
W1115 05:19:38.367414 13527 nvc_info.c:306] missing compat32 library libnvoptix.so
I1115 05:19:38.367609 13527 nvc_info.c:232] selecting /usr/bin/nvidia-smi
I1115 05:19:38.367619 13527 nvc_info.c:232] selecting /usr/bin/nvidia-debugdump
I1115 05:19:38.367628 13527 nvc_info.c:232] selecting /usr/bin/nvidia-persistenced
I1115 05:19:38.367637 13527 nvc_info.c:232] selecting /usr/bin/nvidia-cuda-mps-control
I1115 05:19:38.367647 13527 nvc_info.c:232] selecting /usr/bin/nvidia-cuda-mps-server
I1115 05:19:38.367660 13527 nvc_info.c:369] listing device /dev/nvidiactl
I1115 05:19:38.367663 13527 nvc_info.c:369] listing device /dev/nvidia-uvm
I1115 05:19:38.367667 13527 nvc_info.c:369] listing device /dev/nvidia-uvm-tools
I1115 05:19:38.367671 13527 nvc_info.c:369] listing device /dev/nvidia-modeset
I1115 05:19:38.367685 13527 nvc_info.c:273] listing ipc /run/nvidia-persistenced/socket
W1115 05:19:38.367693 13527 nvc_info.c:277] missing ipc /tmp/nvidia-mps
I1115 05:19:38.367696 13527 nvc_info.c:493] requesting device information with ''
I1115 05:19:38.373812 13527 nvc_info.c:523] listing device /dev/nvidia0 (GPU-e644172f-b35b-feaa-52d2-0290bf9b62e7 at 00000000:01:00.0)
I1115 05:19:38.379545 13527 nvc_info.c:523] listing device /dev/nvidia1 (GPU-4b2c463c-8152-b04c-01ef-f33f3079cba6 at 00000000:03:00.0)
NVRM version:   435.21
CUDA version:   10.1

Device Index:   0
Device Minor:   0
Model:          GeForce GTX 1660
Brand:          GeForce
GPU UUID:       GPU-e644172f-b35b-feaa-52d2-0290bf9b62e7
Bus Location:   00000000:01:00.0
Architecture:   7.5

Device Index:   1
Device Minor:   1
Model:          GeForce GT 640
Brand:          GeForce
GPU UUID:       GPU-4b2c463c-8152-b04c-01ef-f33f3079cba6
Bus Location:   00000000:03:00.0
Architecture:   3.0
I1115 05:19:38.379582 13527 nvc.c:318] shutting down library context
I1115 05:19:38.380110 13529 driver.c:192] terminating driver service
I1115 05:19:38.456407 13527 driver.c:233] driver service terminated successfully

[x] Kernel version from uname -a

Linux PC 5.3.0-22-generic #24-Ubuntu SMP Sat Nov 9 17:34:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[ ] Any relevant kernel output lines from dmesg
[ ] Driver information from nvidia-smi -a
[x] Docker version from docker version

Client:
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.9
 Git commit:        6a30dfca03
 Built:             Mon Sep 16 03:56:22 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.9
  Git commit:       6a30dfca03
  Built:            Wed Sep 11 22:45:55 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10-0ubuntu1
  GitCommit:        
 runc:
  Version:          spec: 1.0.1-dev
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:

[x] NVIDIA packages version from dpkg -l '*nvidia*' _or_ rpm -qa '*nvidia*'

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                             Version         Architecture Description
+++-================================-===============-============-=========================================================
un  libgldispatch0-nvidia            <none>          <none>       (no description available)
ii  libnvidia-cfg1-435:amd64         435.21-0ubuntu2 amd64        NVIDIA binary OpenGL/GLX configuration library
un  libnvidia-cfg1-any               <none>          <none>       (no description available)
un  libnvidia-common                 <none>          <none>       (no description available)
ii  libnvidia-common-435             435.21-0ubuntu2 all          Shared files used by the NVIDIA libraries
ii  libnvidia-compute-435:amd64      435.21-0ubuntu2 amd64        NVIDIA libcompute package
ii  libnvidia-compute-435:i386       435.21-0ubuntu2 i386         NVIDIA libcompute package
ii  libnvidia-container-tools        1.0.5-1         amd64        NVIDIA container runtime library (command-line tools)
ii  libnvidia-container1:amd64       1.0.5-1         amd64        NVIDIA container runtime library
un  libnvidia-decode                 <none>          <none>       (no description available)
ii  libnvidia-decode-435:amd64       435.21-0ubuntu2 amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-decode-435:i386        435.21-0ubuntu2 i386         NVIDIA Video Decoding runtime libraries
un  libnvidia-encode                 <none>          <none>       (no description available)
ii  libnvidia-encode-435:amd64       435.21-0ubuntu2 amd64        NVENC Video Encoding runtime library
ii  libnvidia-encode-435:i386        435.21-0ubuntu2 i386         NVENC Video Encoding runtime library
un  libnvidia-fbc1                   <none>          <none>       (no description available)
ii  libnvidia-fbc1-435:amd64         435.21-0ubuntu2 amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-fbc1-435:i386          435.21-0ubuntu2 i386         NVIDIA OpenGL-based Framebuffer Capture runtime library
un  libnvidia-gl                     <none>          <none>       (no description available)
ii  libnvidia-gl-435:amd64           435.21-0ubuntu2 amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-gl-435:i386            435.21-0ubuntu2 i386         NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
un  libnvidia-ifr1                   <none>          <none>       (no description available)
ii  libnvidia-ifr1-435:amd64         435.21-0ubuntu2 amd64        NVIDIA OpenGL-based Inband Frame Readback runtime library
ii  libnvidia-ifr1-435:i386          435.21-0ubuntu2 i386         NVIDIA OpenGL-based Inband Frame Readback runtime library
un  libnvidia-ml1                    <none>          <none>       (no description available)
un  nvidia-304                       <none>          <none>       (no description available)
un  nvidia-340                       <none>          <none>       (no description available)
un  nvidia-384                       <none>          <none>       (no description available)
un  nvidia-390                       <none>          <none>       (no description available)
un  nvidia-common                    <none>          <none>       (no description available)
ii  nvidia-compute-utils-435         435.21-0ubuntu2 amd64        NVIDIA compute utilities
un  nvidia-container-runtime         <none>          <none>       (no description available)
un  nvidia-container-runtime-hook    <none>          <none>       (no description available)
ii  nvidia-container-toolkit         1.0.5-1         amd64        NVIDIA container runtime hook
ii  nvidia-dkms-435                  435.21-0ubuntu2 amd64        NVIDIA DKMS package
un  nvidia-dkms-kernel               <none>          <none>       (no description available)
ii  nvidia-driver-435                435.21-0ubuntu2 amd64        NVIDIA driver metapackage
un  nvidia-driver-binary             <none>          <none>       (no description available)
un  nvidia-kernel-common             <none>          <none>       (no description available)
ii  nvidia-kernel-common-435         435.21-0ubuntu2 amd64        Shared files used with the kernel module
un  nvidia-kernel-source             <none>          <none>       (no description available)
ii  nvidia-kernel-source-435         435.21-0ubuntu2 amd64        NVIDIA kernel source package
un  nvidia-legacy-304xx-vdpau-driver <none>          <none>       (no description available)
un  nvidia-legacy-340xx-vdpau-driver <none>          <none>       (no description available)
un  nvidia-libopencl1                <none>          <none>       (no description available)
un  nvidia-libopencl1-dev            <none>          <none>       (no description available)
un  nvidia-opencl-icd                <none>          <none>       (no description available)
un  nvidia-persistenced              <none>          <none>       (no description available)
ii  nvidia-prime                     0.8.13          all          Tools to enable NVIDIA's Prime
ii  nvidia-settings                  435.21-0ubuntu2 amd64        Tool for configuring the NVIDIA graphics driver
un  nvidia-settings-binary           <none>          <none>       (no description available)
un  nvidia-smi                       <none>          <none>       (no description available)
un  nvidia-utils                     <none>          <none>       (no description available)
ii  nvidia-utils-435                 435.21-0ubuntu2 amd64        NVIDIA driver support binaries
un  nvidia-vdpau-driver              <none>          <none>       (no description available)
ii  xserver-xorg-video-nvidia-435    435.21-0ubuntu2 amd64        NVIDIA binary Xorg driver

[x] NVIDIA container library version from nvidia-container-cli -V

version: 1.0.5
build date: 2019-09-06T16:59+00:00
build revision: 13b836390888f7b7c7dca115d16d7e28ab15a836
build compiler: x86_64-linux-gnu-gcc-7 7.4.0
build platform: x86_64
build flags: -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -DNDEBUG -std=gnu11 -O2 -g -fdata-sections -ffunction-sections -fstack-protector -fno-strict-aliasing -fvisibility=hidden -Wall -Wextra -Wcast-align -Wpointer-arith -Wmissing-prototypes -Wnonnull -Wwrite-strings -Wlogical-op -Wformat=2 -Wmissing-format-attribute -Winit-self -Wshadow -Wstrict-prototypes -Wunreachable-code -Wconversion -Wsign-conversion -Wno-unknown-warning-option -Wno-format-extra-args -Wno-gnu-alignof-expression -Wl,-zrelro -Wl,-znow -Wl,-zdefs -Wl,--gc-sections