Onnxruntime: Unable to import onnxruntime using the the pre-built container

[diwakar-ravichandran]

Steps to reproduce:
docker pull mcr.microsoft.com/azureml/onnxruntime-tensorrt:latest

nvidia-docker run -it mcr.microsoft.com/azureml/onnxruntime-tensorrt:latest

Error:

python3.7

import onnxruntime

/opt/miniconda/lib/python3.7/site-packages/onnxruntime/capi/_pybind_state.py:13: UserWarning: Cannot load onnxruntime.capi. Error: 'libpython3.7m.so.1.0: cannot open shared object file: No such file or directory'
warnings.warn("Cannot load onnxruntime.capi. Error: '{0}'".format(str(e)))
Traceback (most recent call last):
File "", line 1, in
File "/opt/miniconda/lib/python3.7/site-packages/onnxruntime/__init__.py", line 12, in
from onnxruntime.capi._pybind_state import get_all_providers, get_available_providers, get_device, RunOptions, SessionOptions, set_default_logger_severity, NodeArg, ModelMetadata, GraphOptimizationLevel, ExecutionMode
ImportError: cannot import name 'get_all_providers' from 'onnxruntime.capi._pybind_state' (/opt/miniconda/lib/python3.7/site-packages/onnxruntime/capi/_pybind_state.py)

[jywu-msft]

/opt/miniconda/lib should be added to LD_LIBRARY_PATH (can you try that as a workaround?)
@vinitra , we should add that to the Dockerfile.

[diwakar-ravichandran]

I did try that. While it fixed the first issue. The second one persists on being unable to include get_all_providers

[jywu-msft]

the get_all_providers error message is a bit misleading. that error suggests there is a missing library dependency.
I was able to reproduce your error. however once I added /opt/miniconda/lib to LD_LIBRARY_PATH, I was able to import onnxruntime successfully.

can you paste the output of below commands?

echo $LD_LIBRARY_PATH
ldd /opt/miniconda/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.so

[diwakar-ravichandran]

root@20ff40dbef47:/opt/miniconda/lib# ldd /opt/miniconda/lib/libpython3.7m.so.1.0
linux-vdso.so.1 (0x00007ffc471ed000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b90632000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b90241000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b9003d000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f0b8fe3a000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0b8fc32000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0b8f894000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0b90bb9000)
root@20ff40dbef47:/opt/miniconda/lib# ldd /opt/miniconda/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.so
linux-vdso.so.1 (0x00007fff23661000)
libcublas.so.10 => /usr/lib/x86_64-linux-gnu/libcublas.so.10 (0x00007fee05887000)
libcudnn.so.7 => /usr/lib/x86_64-linux-gnu/libcudnn.so.7 (0x00007fedebc3d000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fedeba39000)
libnvinfer.so.6 => /usr/lib/x86_64-linux-gnu/libnvinfer.so.6 (0x00007fedde6fa000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fedde4f2000)
libcudart.so.10.1 => /usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudart.so.10.1 (0x00007fedde276000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fedddeed000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fedddb4f000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007feddd920000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007feddd708000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007feddd4e9000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feddd0f8000)
/lib64/ld-linux-x86-64.so.2 (0x00007fee0c570000)
libpython3.7m.so.1.0 => not found
libcublasLt.so.10 => /usr/lib/x86_64-linux-gnu/libcublasLt.so.10 (0x00007feddb254000)

[diwakar-ravichandran]

Now this works, thanks a lot 👍