nvidia/cuda:10.1-cudnn7-devel-centos7 miss cublas at some point, which causes our pipeline failure. I pulled the image and tested locally. I found there is no libcublas in /usr/local/cuda/lib64 and that is not the case in nvidia/cuda:10.2-cudnn7-devel-centos7.
You can quick check with following commanad
# run the docker
docker run -d nvidia/cuda:10.1-cudnn7-devel-centos7 sleep 1000000
# get into the bash
docker exec -it xxxxxx /bin/bash
# list the lib64
cd /usr/local/cuda/lib64
ls -alh | grep cublas
Facing same issue on nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04
Most helpful comment
Facing same issue on
nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04