Lxd: Nvidia-smi (No devices where found) if not using root in the container
Required information
- Distribution: Ubuntu
- Distribution version: 16.04
- The output of "lxc info" or if that fails:
- Kernel version: 4.15.0-15-lowlatency
- LXC version: 3.0.0
- LXD version: 3.0.0
- Storage backend in use: N/A
Issue description
Sorry if its a dumb question but, after creates a container with GPU / Cuda capabilites I cannot run nvidia-smi as user ubuntu, but is possible to run as user root
Steps to reproduce
- Setting up the container
lxc launch ubuntu:16.04 my_container
lxc config device add my_container gtx gpu
lxc config set my_container nvidia.runtime true
lxc restart my_container
2.Accessing the nvidia GPU as Root : Here everything works out
lxc exec my_container "nvidia-smi"
or
lxc exec my_container -- su root -c "nvidia-smi"
3.Accessing the nvidia GPU as Ubuntu : No devices were found
lxc exec my_container -- su ubuntu -c "nvidia-smi"
cyberwillis
All 10 comments
Could be a PATH issue or simply that nvidia-smi requires root to initialize cards. //cc @flx42, @3XX0
brauner
on 4 May 2018
👍1
Hi
If I remove this line from the process
lxc config set my_container nvidia.runtime true
and install cuda-toolkit and nvidia-driver inside the container everything works. I was using the earlier method in LXD 2.21, but now in LXD 3.0.0 I tried the new method and did not worked yet.
Also I was following the example of @stgraber from here and I just saw he using root to access it. Anything wrong to try to use ubuntu user ?
cyberwillis
on 4 May 2018
@cyberwillis My guess is that it's got to do with the permissions on some of the /dev files that the nvidia hook setup for you. What does ls -lh /dev look like when nvidia-smi isn't working?
stgraber
on 4 May 2018
👍1
Hi @stgraber
Container:
lxc exec c1 -- su ubuntu
ubuntu@c1:/root$ ls -lh /dev
total 0
crw--w---- 1 root tty 136, 0 May 4 11:40 console
lrwxrwxrwx 1 root root 11 May 4 11:40 core -> /proc/kcore
drwxr-xr-x 2 root root 120 May 4 11:40 dri
lrwxrwxrwx 1 root root 13 May 4 11:40 fd -> /proc/self/fd
crw-rw-rw- 1 nobody nogroup 1, 7 May 4 05:47 full
crw-rw-rw- 1 nobody nogroup 10, 229 May 4 11:40 fuse
drwxr-xr-x 2 root root 40 May 4 11:40 hugepages
lrwxrwxrwx 1 root root 25 May 4 11:40 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root root 28 May 4 11:40 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 nobody nogroup 60 May 4 05:47 lxd
drwxrwxrwt 2 nobody nogroup 40 May 4 05:46 mqueue
drwxr-xr-x 2 root root 60 May 4 11:40 net
crw-rw-rw- 1 nobody nogroup 1, 3 May 4 05:47 null
crw-rw---- 1 root root 195, 0 May 4 11:40 nvidia0
crw-rw---- 1 root root 195, 1 May 4 11:40 nvidia1
crw-rw-rw- 1 nobody nogroup 195, 255 May 4 05:47 nvidiactl
crw-rw-rw- 1 nobody nogroup 239, 0 May 4 05:47 nvidia-uvm
crw-rw-rw- 1 root root 5, 2 May 4 11:40 ptmx
drwxr-xr-x 2 root root 0 May 4 11:40 pts
crw-rw-rw- 1 nobody nogroup 1, 8 May 4 05:47 random
drwxrwxrwt 2 root root 40 May 4 11:40 shm
lrwxrwxrwx 1 root root 15 May 4 11:40 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 May 4 11:40 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 May 4 11:40 stdout -> /proc/self/fd/1
crw-rw-rw- 1 nobody nogroup 5, 0 May 4 05:47 tty
crw-rw-rw- 1 nobody nogroup 1, 9 May 4 05:47 urandom
crw-rw-rw- 1 nobody nogroup 1, 5 May 4 05:47 zero
Host:
$ ls -lh /dev
total 0
crw-r--r-- 1 root root 10, 235 Mai 4 02:47 autofs
drwxr-xr-x 2 root root 420 Mai 4 02:47 block
drwxr-xr-x 2 root root 140 Mai 4 02:47 bsg
crw------- 1 root root 10, 234 Mai 4 02:47 btrfs-control
drwxr-xr-x 3 root root 60 Mai 4 02:46 bus
lrwxrwxrwx 1 root root 3 Mai 4 02:47 cdrom -> sr0
lrwxrwxrwx 1 root root 3 Mai 4 02:47 cdrw -> sr0
drwxr-xr-x 2 root root 4,9K Mai 4 02:47 char
crw------- 1 root root 5, 1 Mai 4 02:47 console
lrwxrwxrwx 1 root root 11 Mai 4 02:46 core -> /proc/kcore
drwxr-xr-x 2 root root 60 Mai 4 02:46 cpu
crw------- 1 root root 10, 59 Mai 4 02:47 cpu_dma_latency
crw------- 1 root root 10, 203 Mai 4 02:47 cuse
drwxr-xr-x 7 root root 140 Mai 4 02:47 disk
drwxr-xr-x 2 root root 120 Mai 4 02:46 dri
lrwxrwxrwx 1 root root 3 Mai 4 02:47 dvd -> sr0
lrwxrwxrwx 1 root root 3 Mai 4 02:47 dvdrw -> sr0
crw------- 1 root root 10, 61 Mai 4 02:47 ecryptfs
lrwxrwxrwx 1 root root 13 Mai 4 02:46 fd -> /proc/self/fd
crw-rw-rw- 1 root root 1, 7 Mai 4 02:47 full
crw-rw-rw- 1 root root 10, 229 Mai 4 08:40 fuse
crw------- 1 root root 243, 0 Mai 4 02:47 fw0
crw------- 1 root root 240, 0 Mai 4 02:47 hidraw0
crw------- 1 root root 240, 1 Mai 4 02:47 hidraw1
crw------- 1 root root 240, 2 Mai 4 02:47 hidraw2
crw------- 1 root root 240, 3 Mai 4 02:47 hidraw3
crw------- 1 root root 10, 228 Mai 4 02:47 hpet
drwxr-xr-x 2 root root 0 Mai 4 02:47 hugepages
crw------- 1 root root 10, 183 Mai 4 02:47 hwrng
crw------- 1 root root 89, 0 Mai 4 02:47 i2c-0
crw------- 1 root root 89, 1 Mai 4 02:47 i2c-1
crw------- 1 root root 89, 2 Mai 4 02:47 i2c-2
crw------- 1 root root 89, 3 Mai 4 02:47 i2c-3
crw------- 1 root root 89, 4 Mai 4 02:47 i2c-4
crw------- 1 root root 89, 5 Mai 4 02:47 i2c-5
crw------- 1 root root 89, 6 Mai 4 02:47 i2c-6
lrwxrwxrwx 1 root root 25 Mai 4 02:47 initctl -> /run/systemd/initctl/fifo
drwxr-xr-x 4 root root 580 Mai 4 02:47 input
crw-r--r-- 1 root root 1, 11 Mai 4 02:47 kmsg
crw-rw----+ 1 root root 10, 232 Mai 4 02:47 kvm
drwxr-xr-x 2 root root 60 Mai 4 02:46 lightnvm
lrwxrwxrwx 1 root root 28 Mai 4 02:47 log -> /run/systemd/journal/dev-log
brw-rw---- 1 root disk 7, 0 Mai 4 02:47 loop0
brw-rw---- 1 root disk 7, 1 Mai 4 02:47 loop1
brw-rw---- 1 root disk 7, 2 Mai 4 02:47 loop2
brw-rw---- 1 root disk 7, 3 Mai 4 02:47 loop3
brw-rw---- 1 root disk 7, 4 Mai 4 02:47 loop4
brw-rw---- 1 root disk 7, 5 Mai 4 02:47 loop5
brw-rw---- 1 root disk 7, 6 Mai 4 02:47 loop6
brw-rw---- 1 root disk 7, 7 Mai 4 02:47 loop7
crw-rw---- 1 root disk 10, 237 Mai 4 02:47 loop-control
drwxr-xr-x 2 root root 60 Mai 4 02:46 mapper
crw------- 1 root root 10, 227 Mai 4 02:47 mcelog
crw------- 1 root root 238, 0 Mai 4 02:47 media0
crw-r----- 1 root kmem 1, 1 Mai 4 02:47 mem
crw------- 1 root root 10, 56 Mai 4 02:47 memory_bandwidth
drwxrwxrwt 2 root root 40 Mai 4 02:46 mqueue
drwxr-xr-x 2 root root 60 Mai 4 02:46 net
crw------- 1 root root 10, 58 Mai 4 02:47 network_latency
crw------- 1 root root 10, 57 Mai 4 02:47 network_throughput
crw-rw-rw- 1 root root 1, 3 Mai 4 02:47 null
crw-rw-rw- 1 root root 195, 0 Mai 4 02:47 nvidia0
crw-rw-rw- 1 root root 195, 1 Mai 4 02:47 nvidia1
crw-rw-rw- 1 root root 195, 255 Mai 4 02:47 nvidiactl
crw-rw-rw- 1 root root 195, 254 Mai 4 02:47 nvidia-modeset
crw-rw-rw- 1 root root 239, 0 Mai 4 02:47 nvidia-uvm
crw-r----- 1 root kmem 1, 4 Mai 4 02:47 port
crw------- 1 root root 108, 0 Mai 4 02:47 ppp
crw------- 1 root root 10, 1 Mai 4 02:47 psaux
crw-rw-rw- 1 root tty 5, 2 Mai 4 09:53 ptmx
drwxr-xr-x 2 root root 0 Mai 4 02:46 pts
crw-rw-rw- 1 root root 1, 8 Mai 4 02:47 random
crw-rw-r--+ 1 root netdev 10, 62 Mai 4 02:47 rfkill
lrwxrwxrwx 1 root root 4 Mai 4 02:47 rtc -> rtc0
crw------- 1 root root 249, 0 Mai 4 02:47 rtc0
brw-rw---- 1 root disk 8, 0 Mai 4 02:47 sda
brw-rw---- 1 root disk 8, 1 Mai 4 02:47 sda1
brw-rw---- 1 root disk 8, 2 Mai 4 02:47 sda2
brw-rw---- 1 root disk 8, 5 Mai 4 02:47 sda5
brw-rw---- 1 root disk 8, 16 Mai 4 02:47 sdb
brw-rw---- 1 root disk 8, 17 Mai 4 02:47 sdb1
brw-rw---- 1 root disk 8, 32 Mai 4 02:47 sdc
brw-rw---- 1 root disk 8, 33 Mai 4 02:47 sdc1
brw-rw---- 1 root disk 8, 48 Mai 4 02:47 sdd
brw-rw---- 1 root disk 8, 49 Mai 4 02:47 sdd1
crw-rw---- 1 root disk 21, 0 Mai 4 02:47 sg0
crw-rw---- 1 root disk 21, 1 Mai 4 02:47 sg1
crw-rw---- 1 root disk 21, 2 Mai 4 02:47 sg2
crw-rw---- 1 root disk 21, 3 Mai 4 02:47 sg3
crw-rw----+ 1 root cdrom 21, 4 Mai 4 02:47 sg4
drwxrwxrwt 2 root root 240 Mai 4 09:53 shm
crw------- 1 root root 10, 231 Mai 4 02:47 snapshot
drwxr-xr-x 4 root root 520 Mai 4 02:47 snd
brw-rw----+ 1 root cdrom 11, 0 Mai 4 02:47 sr0
lrwxrwxrwx 1 root root 15 Mai 4 02:46 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Mai 4 02:46 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Mai 4 02:46 stdout -> /proc/self/fd/1
crw-rw-rw- 1 root tty 5, 0 Mai 4 02:47 tty
crw--w---- 1 root tty 4, 0 Mai 4 02:47 tty0
crw--w---- 1 root tty 4, 1 Mai 4 02:47 tty1
crw--w---- 1 root tty 4, 10 Mai 4 02:47 tty10
crw--w---- 1 root tty 4, 11 Mai 4 02:47 tty11
crw--w---- 1 root tty 4, 12 Mai 4 02:47 tty12
crw--w---- 1 root tty 4, 13 Mai 4 02:47 tty13
crw--w---- 1 root tty 4, 14 Mai 4 02:47 tty14
crw--w---- 1 root tty 4, 15 Mai 4 02:47 tty15
crw--w---- 1 root tty 4, 16 Mai 4 02:47 tty16
crw--w---- 1 root tty 4, 17 Mai 4 02:47 tty17
crw--w---- 1 root tty 4, 18 Mai 4 02:47 tty18
crw--w---- 1 root tty 4, 19 Mai 4 02:47 tty19
crw--w---- 1 root tty 4, 2 Mai 4 02:47 tty2
crw--w---- 1 root tty 4, 20 Mai 4 02:47 tty20
crw--w---- 1 root tty 4, 21 Mai 4 02:47 tty21
crw--w---- 1 root tty 4, 22 Mai 4 02:47 tty22
crw--w---- 1 root tty 4, 23 Mai 4 02:47 tty23
crw--w---- 1 root tty 4, 24 Mai 4 02:47 tty24
crw--w---- 1 root tty 4, 25 Mai 4 02:47 tty25
crw--w---- 1 root tty 4, 26 Mai 4 02:47 tty26
crw--w---- 1 root tty 4, 27 Mai 4 02:47 tty27
crw--w---- 1 root tty 4, 28 Mai 4 02:47 tty28
crw--w---- 1 root tty 4, 29 Mai 4 02:47 tty29
crw--w---- 1 root tty 4, 3 Mai 4 02:47 tty3
crw--w---- 1 root tty 4, 30 Mai 4 02:47 tty30
crw--w---- 1 root tty 4, 31 Mai 4 02:47 tty31
crw--w---- 1 root tty 4, 32 Mai 4 02:47 tty32
crw--w---- 1 root tty 4, 33 Mai 4 02:47 tty33
crw--w---- 1 root tty 4, 34 Mai 4 02:47 tty34
crw--w---- 1 root tty 4, 35 Mai 4 02:47 tty35
crw--w---- 1 root tty 4, 36 Mai 4 02:47 tty36
crw--w---- 1 root tty 4, 37 Mai 4 02:47 tty37
crw--w---- 1 root tty 4, 38 Mai 4 02:47 tty38
crw--w---- 1 root tty 4, 39 Mai 4 02:47 tty39
crw--w---- 1 root tty 4, 4 Mai 4 02:47 tty4
crw--w---- 1 root tty 4, 40 Mai 4 02:47 tty40
crw--w---- 1 root tty 4, 41 Mai 4 02:47 tty41
crw--w---- 1 root tty 4, 42 Mai 4 02:47 tty42
crw--w---- 1 root tty 4, 43 Mai 4 02:47 tty43
crw--w---- 1 root tty 4, 44 Mai 4 02:47 tty44
crw--w---- 1 root tty 4, 45 Mai 4 02:47 tty45
crw--w---- 1 root tty 4, 46 Mai 4 02:47 tty46
crw--w---- 1 root tty 4, 47 Mai 4 02:47 tty47
crw--w---- 1 root tty 4, 48 Mai 4 02:47 tty48
crw--w---- 1 root tty 4, 49 Mai 4 02:47 tty49
crw--w---- 1 root tty 4, 5 Mai 4 02:47 tty5
crw--w---- 1 root tty 4, 50 Mai 4 02:47 tty50
crw--w---- 1 root tty 4, 51 Mai 4 02:47 tty51
crw--w---- 1 root tty 4, 52 Mai 4 02:47 tty52
crw--w---- 1 root tty 4, 53 Mai 4 02:47 tty53
crw--w---- 1 root tty 4, 54 Mai 4 02:47 tty54
crw--w---- 1 root tty 4, 55 Mai 4 02:47 tty55
crw--w---- 1 root tty 4, 56 Mai 4 02:47 tty56
crw--w---- 1 root tty 4, 57 Mai 4 02:47 tty57
crw--w---- 1 root tty 4, 58 Mai 4 02:47 tty58
crw--w---- 1 root tty 4, 59 Mai 4 02:47 tty59
crw--w---- 1 root tty 4, 6 Mai 4 02:47 tty6
crw--w---- 1 root tty 4, 60 Mai 4 02:47 tty60
crw--w---- 1 root tty 4, 61 Mai 4 02:47 tty61
crw--w---- 1 root tty 4, 62 Mai 4 02:47 tty62
crw--w---- 1 root tty 4, 63 Mai 4 02:47 tty63
crw--w---- 1 root tty 4, 7 Mai 4 02:47 tty7
crw--w---- 1 root tty 4, 8 Mai 4 02:47 tty8
crw--w---- 1 root tty 4, 9 Mai 4 02:47 tty9
crw------- 1 root root 5, 3 Mai 4 02:47 ttyprintk
crw-rw---- 1 root dialout 4, 64 Mai 4 02:47 ttyS0
crw-rw---- 1 root dialout 4, 65 Mai 4 02:47 ttyS1
crw-rw---- 1 root dialout 4, 74 Mai 4 02:47 ttyS10
crw-rw---- 1 root dialout 4, 75 Mai 4 02:47 ttyS11
crw-rw---- 1 root dialout 4, 76 Mai 4 02:47 ttyS12
crw-rw---- 1 root dialout 4, 77 Mai 4 02:47 ttyS13
crw-rw---- 1 root dialout 4, 78 Mai 4 02:47 ttyS14
crw-rw---- 1 root dialout 4, 79 Mai 4 02:47 ttyS15
crw-rw---- 1 root dialout 4, 80 Mai 4 02:47 ttyS16
crw-rw---- 1 root dialout 4, 81 Mai 4 02:47 ttyS17
crw-rw---- 1 root dialout 4, 82 Mai 4 02:47 ttyS18
crw-rw---- 1 root dialout 4, 83 Mai 4 02:47 ttyS19
crw-rw---- 1 root dialout 4, 66 Mai 4 02:47 ttyS2
crw-rw---- 1 root dialout 4, 84 Mai 4 02:47 ttyS20
crw-rw---- 1 root dialout 4, 85 Mai 4 02:47 ttyS21
crw-rw---- 1 root dialout 4, 86 Mai 4 02:47 ttyS22
crw-rw---- 1 root dialout 4, 87 Mai 4 02:47 ttyS23
crw-rw---- 1 root dialout 4, 88 Mai 4 02:47 ttyS24
crw-rw---- 1 root dialout 4, 89 Mai 4 02:47 ttyS25
crw-rw---- 1 root dialout 4, 90 Mai 4 02:47 ttyS26
crw-rw---- 1 root dialout 4, 91 Mai 4 02:47 ttyS27
crw-rw---- 1 root dialout 4, 92 Mai 4 02:47 ttyS28
crw-rw---- 1 root dialout 4, 93 Mai 4 02:47 ttyS29
crw-rw---- 1 root dialout 4, 67 Mai 4 02:47 ttyS3
crw-rw---- 1 root dialout 4, 94 Mai 4 02:47 ttyS30
crw-rw---- 1 root dialout 4, 95 Mai 4 02:47 ttyS31
crw-rw---- 1 root dialout 4, 68 Mai 4 02:47 ttyS4
crw-rw---- 1 root dialout 4, 69 Mai 4 02:47 ttyS5
crw-rw---- 1 root dialout 4, 70 Mai 4 02:47 ttyS6
crw-rw---- 1 root dialout 4, 71 Mai 4 02:47 ttyS7
crw-rw---- 1 root dialout 4, 72 Mai 4 02:47 ttyS8
crw-rw---- 1 root dialout 4, 73 Mai 4 02:47 ttyS9
crw------- 1 root root 10, 239 Mai 4 02:47 uhid
crw------- 1 root root 10, 223 Mai 4 02:47 uinput
crw-rw-rw- 1 root root 1, 9 Mai 4 02:47 urandom
crw------- 1 root root 10, 240 Mai 4 02:47 userio
drwxr-xr-x 4 root root 80 Mai 4 02:47 v4l
crw------- 1 root root 10, 55 Mai 4 02:47 vboxdrv
crw-rw-rw- 1 root root 10, 54 Mai 4 02:47 vboxdrvu
crw------- 1 root root 10, 53 Mai 4 02:47 vboxnetctl
drwxr-x--- 5 root vboxusers 100 Mai 4 02:47 vboxusb
crw-rw---- 1 root tty 7, 0 Mai 4 02:47 vcs
crw-rw---- 1 root tty 7, 1 Mai 4 02:47 vcs1
crw-rw---- 1 root tty 7, 2 Mai 4 02:47 vcs2
crw-rw---- 1 root tty 7, 3 Mai 4 02:47 vcs3
crw-rw---- 1 root tty 7, 4 Mai 4 02:47 vcs4
crw-rw---- 1 root tty 7, 5 Mai 4 02:47 vcs5
crw-rw---- 1 root tty 7, 6 Mai 4 02:47 vcs6
crw-rw---- 1 root tty 7, 7 Mai 4 02:47 vcs7
crw-rw---- 1 root tty 7, 128 Mai 4 02:47 vcsa
crw-rw---- 1 root tty 7, 129 Mai 4 02:47 vcsa1
crw-rw---- 1 root tty 7, 130 Mai 4 02:47 vcsa2
crw-rw---- 1 root tty 7, 131 Mai 4 02:47 vcsa3
crw-rw---- 1 root tty 7, 132 Mai 4 02:47 vcsa4
crw-rw---- 1 root tty 7, 133 Mai 4 02:47 vcsa5
crw-rw---- 1 root tty 7, 134 Mai 4 02:47 vcsa6
crw-rw---- 1 root tty 7, 135 Mai 4 02:47 vcsa7
drwxr-xr-x 2 root root 60 Mai 4 02:47 vfio
crw------- 1 root root 10, 63 Mai 4 02:47 vga_arbiter
crw------- 1 root root 10, 137 Mai 4 02:47 vhci
crw------- 1 root root 10, 238 Mai 4 02:47 vhost-net
crw------- 1 root root 10, 241 Mai 4 02:47 vhost-vsock
crw-rw----+ 1 root video 81, 0 Mai 4 02:47 video0
crw-rw-rw- 1 root root 10, 52 Mai 4 02:47 vmci
crw-rw-rw- 1 root root 10, 51 Mai 4 02:47 vsock
crw-rw-rw- 1 root root 1, 5 Mai 4 02:47 zero
Any advice in how can I solve this issue ?
thanks
cyberwillis
on 4 May 2018
Hi again @stgraber
I patched inside the container by adding permissions :
sudo chmod 666 /dev/nvidia0
sudo chmod 666 /dev/nvidia1
and it worked as ubuntu user now.
its the correct way to do it ?
Thanks in advance
[updated]
cyberwillis
on 4 May 2018
I just detected that after restart the container the problem returns.
For the moment I fix it by setting it permanently on startup of each container I create. In case anybody else needs it:
/etc/rc.local
#!/bin/sh -e
chmod 666 /dev/nvidia0
chmod 666 /dev/nvidia1
exit 0
Looking forward to see a official fix for it.
[updated]
cyberwillis
on 4 May 2018
👍1
666 would be a better permission here.
@brauner is that our fault or nvidia's? If it's our fault, can you send a branch?
I think we should have the mode match that of the host for any of those files.
stgraber
on 4 May 2018
I suspect it's yours since we don't mount the device nodes in LXD, only in LXC :)
If the device nodes are recreated inside the container, they usually need to match the permissions enforced by the kernel modules:
cat /proc/driver/nvidia/params
DeviceFileUID: 0
DeviceFileGID: 0
DeviceFileMode: 438 # 0666 in octal
Note that these permissions are from the init userns perspective, udev can also do something different on top of that...
If you look at the device nodes mounted by our hook, they are correct:
crw-rw-rw- 1 nobody nogroup 195, 255 May 4 05:47 nvidiactl
crw-rw-rw- 1 nobody nogroup 239, 0 May 4 05:47 nvidia-uvm
Same should be done with the /dev/nvidia[N]
Obviously your UID/GID remapped inside the container then needs permission to access it.
So you most likely want to configure DeviceFileMode or udev with the right permissions for your containers (see also How and when are the NVIDIA device files created?).
3XX0
on 4 May 2018
Yeah, had a chat with @brauner and he said we're using our default device passthrough code, which unless you're specifying your own mode.
// Get the device mode
mode := os.FileMode(0660)
if m["mode"] != "" {
tmp, err := deviceModeOct(m["mode"])
if err != nil {
return nil, fmt.Errorf("Bad mode %s in device %s", m["mode"], m["path"])
}
mode = os.FileMode(tmp)
}
What we should do when passing devices automatically (so for GPU, USB and Infiniband) is replicate the permissions found on the host.
@brauner assigning to you
stgraber
on 4 May 2018
Just detected another issue
I don't know if it's related too
The container I was working on had a folder called /projects mapped inside the container. However because the folder was unprivileged all the gid and uid was as nobody nobody
and trying to make writing to that folder wasn't possible
I tried to convert the container to provileged one by:
lxc config set c1 security.privileged true
lxc config set c1 volatile.idmap.next 1000
lxc restart c1
However it conflicts with the fact I had used this line before
lxc config set c1 nvidia.runtime true
Unless I set nvidia.runtime to false I cannot start the container as privileged.
output:
Error: Failed to run: /snap/lxd/current/bin/lxd forkstart c1 /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/c1/lxc.conf: Try lxc info --show-log c1 for more info
/var/snap/lxd/common/lxd/logs/c1/lxc.conf:
Name: c1
Remote: unix://
Architecture: x86_64
Created: 2018/05/05 12:06 UTC
Status: Stopped
Type: persistent
Profiles: default
Log:
lxc 20180505120800.640 ERROR lxc_conf - conf.c:run_buffer:347 - Script exited with status 1
lxc 20180505120800.640 ERROR lxc_conf - conf.c:lxc_setup:3391 - Failed to run mount hooks
lxc 20180505120800.640 ERROR lxc_start - start.c:do_start:1198 - Failed to setup container "c1"
lxc 20180505120800.640 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5)
lxc 20180505120800.663 ERROR lxc_start - start.c:__lxc_start:1866 - Failed to spawn container "c1"
lxc 20180505120800.663 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:824 - Received container state "ABORTING" instead of "RUNNING"
lxc 20180505120800.665 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:130 - Connection reset by peer - Failed to receive response for command "get_cgroup"
configuration:
$ lxc config show c1 --expanded
architecture: x86_64
config:
environment.http_proxy: ""
image.architecture: amd64
image.description: ubuntu 16.04 LTS amd64 (release) (20180427)
image.label: release
image.os: ubuntu
image.release: xenial
image.serial: "20180427"
image.version: "16.04"
limits.cpu: "4"
nvidia.runtime: "true"
security.privileged: "true"
user.network_mode: ""
volatile.base_image: 353b1a2c367ec983fd9d1532171618cd967e96d77a06f6b6e024c39ec010e8d7
volatile.eth0.hwaddr: 00:16:3e:df:5c:49
volatile.eth0.name: eth0
volatile.idmap.base: "0"
volatile.idmap.next: "1000"
volatile.last_state.idmap: '[]'
volatile.last_state.power: STOPPED
devices:
eth0:
nictype: bridged
parent: lxdbr0
type: nic
gtx:
type: gpu
projects:
path: /projects
source: /home/cyberwillis/projects
type: disk
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
Was this page helpful?
0 / 5 - 0 ratings
Related issues
rrva
路
5Comments
rrva
路
5Comments
error: Failed to destroy ZFS filesystem: cannot destroy 'lxd/containers/lc3': dataset already exists
tebanep
路
5Comments
fwaggle
路
4Comments
simos
路
3Comments