Onnxruntime: Lack of support for Windows/CUDA in Python 3.8 and 3.9

[evrial]

I hate to say but Python 3.8 released on 27-Jan-2018, (nearly two years ago) still Microsoft didn't manage to make support. In my humble opinion thats pathetic. Would you guys please update us with your development progress on this path? Thanks and God bless you all and your families. I hope you have a peaceful President election. Please wear masks and stay safe.

[faxu]

Python 3.8 support is available for everything except Windows GPU and Linux ARM. Those will be added in the next release. What setup do you have?

[snnn]

There is a big incompatible change in python 3.8, we haven't find a way to solve it yet.

python/cpython@2438cdf
https://bugs.python.org/issue36085

[honglei]

How about suppport on Python3.9？ Python 3.9.0 is now available, and you can already test 3.10.0a1!

[snnn]

Sorry, it is not started yet.

[ivanst0]

There are two distinct issues which probably won't be fixed at same time:
1) Lack of support for Windows/CUDA in Python 3.8 (i.e. onnxruntime_gpu-*-cp38-cp38-win_amd64.whl not available on PyPI).
2) Lack of support for Python 3.9 in general.

Could you please change the title of this issue to make it clear what it refers to and open a new issue for the other one (if there is no existing issue covering it)?

[evrial]

Thanks for clarification, updated as you suggested. I wish the complete support of the recent Python will be a new norm with Guido in your team.

[ivanst0]

@snnn, would it be possible to invoke os.add_dll_directory() (for Python 3.8 and newer) from __init__.py after finding CUDA location from environment variables? A robust implementation would also require exposing CUDA version to Python code.

Has anyone looked (or plans to look) at this?

[snnn]

I don't have any good ideas. Welcome to contribute yours!

[evrial]

Maybe ask the user to put CUDA location in some config file as fallback if all else fails

[honglei]

where nvcc

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\nvcc.exe

nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:32:27_Pacific_Daylight_Time_2019
Cuda compilation tools, release 10.2, V10.2.89

[ivanst0]

OK, I'll try to come up with a fix.

[ivanst0]

Here is a potential solution: https://github.com/microsoft/onnxruntime/pull/5956
Although it doesn't handle all CUDA versions, this might be already sufficient to enable Python 3.8 build in CI and publish a Windows version of onnxruntime-gpu package for Python 3.8.

[ivanst0]

After PR https://github.com/microsoft/onnxruntime/pull/5956 onnxruntime-gpu package can be built for Python 3.8 and Python 3.9, so I think this issue can be closed now.

The remaining issues are tracked in
https://github.com/microsoft/onnxruntime/issues/5963 - onnxruntime-gpu package not available for Python 3.8 on Windows
https://github.com/microsoft/onnxruntime/issues/5962 - onnxruntime-gpu built for Python 3.8/Windows doesn't support CUDA versions other than 10.2