Onnxruntime: About CUDA's version

[eekarot]

Can I use version 11 or above？

[harshithapv]

If you are asking about whether you can build onnxruntime with CUDA 11.1. This has been tested and you should be able to use CUDA 11.1.

[pommedeterresautee]

With CUDA 11.1, tests fail :-(

...
git clone ...
git checkout 718ca7f # (version 1.6.0)
CUDACXX=/usr/local/cuda-11.1/bin/nvcc ./build.sh --config Release --build_wheel --parallel --use_cuda --cuda_home /usr/local/cuda-11.1 --cudnn_home /usr/lib/x86_64-linux-gnu/

Generate:

...
[----------] 8 tests from LinearRegressorTest/LinearRegressorTest (3 ms total)
1: 
1: [----------] 5 tests from CApiTestWithProviders/CApiTestWithProvider
1: [ RUN      ] CApiTestWithProviders/CApiTestWithProvider.simple/0
1: Running simple inference with default provider
1: [       OK ] CApiTestWithProviders/CApiTestWithProvider.simple/0 (1 ms)
1: [ RUN      ] CApiTestWithProviders/CApiTestWithProvider.simple/1
1: Running simple inference with cuda provider
1: [       OK ] CApiTestWithProviders/CApiTestWithProvider.simple/1 (3 ms)
1: [ RUN      ] CApiTestWithProviders/CApiTestWithProvider.simple/2
1: [       OK ] CApiTestWithProviders/CApiTestWithProvider.simple/2 (0 ms)
1: [ RUN      ] CApiTestWithProviders/CApiTestWithProvider.simple/3
1: [       OK ] CApiTestWithProviders/CApiTestWithProvider.simple/3 (0 ms)
1: [ RUN      ] CApiTestWithProviders/CApiTestWithProvider.simple/4
1: Running simple inference with default provider
1: [       OK ] CApiTestWithProviders/CApiTestWithProvider.simple/4 (0 ms)
1: [----------] 5 tests from CApiTestWithProviders/CApiTestWithProvider (5 ms total)
1: 
1: [----------] Global test environment tear-down
1: [==========] 2431 tests from 198 test suites ran. (27154 ms total)
1: [  PASSED  ] 2418 tests.
1: [  FAILED  ] 13 tests, listed below:
1: [  FAILED  ] QAttentionTest.QAttentionBatch1
1: [  FAILED  ] QAttentionTest.QAttentionBatch1_Float16
1: [  FAILED  ] QAttentionTest.QAttentionBatch2
1: [  FAILED  ] QAttentionTest.QAttentionMaskPartialSequence
1: [  FAILED  ] QAttentionTest.QAttentionMaskExceedSequence
1: [  FAILED  ] QAttentionTest.QAttentionNoMaskIndex
1: [  FAILED  ] QAttentionTest.QAttentionUnidirectional_CUDA
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t_A_ND
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t_B_ND
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t_A_ND_B_ND
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t_A_Has_Zero_Point
1: [  FAILED  ] MatmulIntegerOpTest.MatMulInteger_int8_t_No_Zero_Point
1: 
1: 13 FAILED TESTS
1:   YOU HAVE 6 DISABLED TESTS
1: 
1/3 Test #1: onnxruntime_test_all .............***Failed   27.70 sec
test 2
    Start 2: onnx_test_pytorch_converted

2: Test command: /mnt/workspace/onnxruntime/build/Linux/Release/onnx_test_runner "/mnt/workspace/onnxruntime/cmake/external/onnx/onnx/backend/test/data/pytorch-converted"
2: Test timeout computed to be: 3600
2: 2020-12-19 23:32:42.739477607 [E:onnxruntime:Default, testcase_driver.cc:39 RunParallel] Running tests in parallel: at most 12 models at any time
2: 2020-12-19 23:32:42.781376554 [E:onnxruntime:Default, testcase_driver.cc:61 RunModelsAsync] Running tests finished. Generating report
2: result: 
2:  Models: 57
2:  Total test cases: 57
2:      Succeeded: 57
2:      Not implemented: 0
2:      Failed: 0
2:  Stats by Operator type:
2:      Not implemented(0): 
2:      Failed:
2: Failed Test Cases:
2/3 Test #2: onnx_test_pytorch_converted ......   Passed    0.07 sec
test 3
    Start 3: onnx_test_pytorch_operator

3: Test command: /mnt/workspace/onnxruntime/build/Linux/Release/onnx_test_runner "/mnt/workspace/onnxruntime/cmake/external/onnx/onnx/backend/test/data/pytorch-operator"
3: Test timeout computed to be: 3600
3: 2020-12-19 23:32:42.811606633 [E:onnxruntime:Default, testcase_driver.cc:39 RunParallel] Running tests in parallel: at most 12 models at any time
3: 2020-12-19 23:32:42.833771281 [E:onnxruntime:Default, testcase_driver.cc:61 RunModelsAsync] Running tests finished. Generating report
3: result: 
3:  Models: 24
3:  Total test cases: 24
3:      Succeeded: 24
3:      Not implemented: 0
3:      Failed: 0
3:  Stats by Operator type:
3:      Not implemented(0): 
3:      Failed:
3: Failed Test Cases:
3/3 Test #3: onnx_test_pytorch_operator .......   Passed    0.05 sec

67% tests passed, 1 tests failed out of 3

Total Test time (real) =  27.84 sec

The following tests FAILED:
      1 - onnxruntime_test_all (Failed)
Errors while running CTest
Traceback (most recent call last):
  File "/home/geantvert/workspace/onnxruntime/tools/ci_build/build.py", line 2088, in <module>
    sys.exit(main())
  File "/home/geantvert/workspace/onnxruntime/tools/ci_build/build.py", line 2026, in main
    run_onnxruntime_tests(args, source_dir, ctest_path, build_dir, configs)
  File "/home/geantvert/workspace/onnxruntime/tools/ci_build/build.py", line 1450, in run_onnxruntime_tests
    run_subprocess(ctest_cmd, cwd=cwd, dll_path=dll_path)
  File "/home/geantvert/workspace/onnxruntime/tools/ci_build/build.py", line 523, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/mnt/workspace/onnxruntime/tools/python/util/run.py", line 39, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/ctest', '--build-config', 'Release', '--verbose', '--timeout', '3600']' returned non-zero exit status 8.