The docker is using docker-ce to replace docker-engine, and nvidia-docker seems not support it.
Which version of nvidia-docker do you have? You need to upgrade to 1.0.1
I encountered the same issue. I have upgraded from ubuntu 14.04 to ubuntu 16.04.
In ubuntu 14.04 with docker-engine, nvidia-docker works fine.
But now in ubuntu 16.04 with docker-ce, it came wrong.
➜ ~ docker -v
Docker version 17.03.1-ce, build c6d412e
➜ ~ nvidia-docker -v
Docker version 17.03.1-ce, build c6d412e
➜ ~ nvidia-docker run --rm nvidia/cuda nvidia-smi
container_linux.go:247: starting container process caused "exec: \"nvidia-smi\": executable file not found in $PATH"
docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"nvidia-smi\": executable file not found in $PATH".
Yes, I'm using nvidia-docker 1.0.1.
I have downgraded docker from 17.03.1-ce to 17.03.0-ce. The error already exists.
I really have no idea what the problem is.
@ksnzh take a look at this thread:
https://github.com/NVIDIA/nvidia-docker/issues/349
It might help.
@flx42 docker volume rm -f nvidia_driver_375.39 helped me.
I manually removed the volume and retried. It seems everything goes okey.
Most helpful comment
@flx42
docker volume rm -f nvidia_driver_375.39helped me.I manually removed the volume and retried. It seems everything goes okey.