Bentoml: onnxruntime-gpu not recognized as a valid onnxruntime python package

[BorhenJlidi]

Describe the bug
When saving a model for inference (ONNX resnet 50 example), the following warning is outputted :

WARNING - pip package requirementonnxruntimenot found in current python environment

This can confuse BentoML users and can possibly drive off new users.

To Reproduce

1. Install onnxruntime-gpu (doesn't matter prebuilt or compiled from source)
2. Run onnx resnet50 example

Expected behavior
It shouldn't output such warning

Screenshots/Logs
WARNING - pip package requirementonnxruntimenot found in current python environment

Environment:

• OS: Ubuntu 20.04
• Python Version 3.8.5
• BentoML Version 0.92
• onnxruntime 1.5.2 (onnxruntime-gpu)

[yubozhao]

@BorhenJlidi Thanks for reporting. Right now, it only supports non-gpu version of onnxruntime. You are right, we should make it more clear to the users.
I think I will 1. make the message more clear and 2. start researching add support for onnxruntime-gpu.

[BorhenJlidi]

@yubozhao Just out of curiosity, when you say it it only supports non-gpu version of onnxruntime does that mean that it won't work at all or just output the warning?

[parano]

@BorhenJlidi - it's a mistake on our end, BentoML's OnnxModelArtifact assumed any user that is using ONNX will be relying on the onnxruntime PyPI package whereas some users are using the alternative onnxruntime-gpu.

A workaround for you now, is to add onnxruntime-gpu to the @env definition in your BentoService class:

@env(infer_pip_packages=True, pip_packages=["onnxruntime-gpu"])

I will create a github issue for tracking a fix to this issue. Thanks for reporting it! @BorhenJlidi

[BorhenJlidi]

@parano That's a nice workaround, Thanks! Keep up the amazing work !