Onnxruntime: ONNX Runtime and ONNXIFI

[JammyZhou]

Hi,

I'm trying to understand the relationship between ONNX Runtime and ONNXIFI. ONNXIFI was released as part of ONNX 1.3, and Glow backend is already implemented by Facebook, nGraph also has some plan to support it. And for ONNX runtime, we have different execution providers (such as nGraph, CUDA, TensorRT, etc) to support various hardware accelerators. To avoid some kind of fragmentation in the ONNX ecosystem, how ONNX runtime and ONNXIFI can work together?

[linkerzhang]

As you mentioned, various hardware accelerators are actively working with ONNX runtime on the integration, say, nGraph, TensorRT, etc. Positive feedbacks have been received about the integration API designed so far. Meanwhile, we're still working together to keep tuning the API till it's good and general for all partners. It (the API) may be considered to be pushed into ONNX as ONNXIFI v2 if needed, for example.

[yuanbyu]

The model format (onnx/onnx.proto) and the set of operators (onnx/defs) constitute the ONNX standard. ONNX Runtime tries to provide a simple and expressive API to interact with all kinds of backends.

[JammyZhou]

@linkerzhang @yuanbyu Thanks for your reply and happy holiday! I agree that ONNX Runtime has good momentum and I like the idea of its full ONNX spec compliance. I think the goal of ONNX organization in some extent is to reduce the 'fragmentation' in the ecosystem, and we shouldn't re-invent the wheel for similar 'standards' to support accelerators from multiple vendors (to me, ONNXIFI and ONNX Runtime are quite similar, and please correct if my understanding is wrong). Do we have some forum in ONNX side to discuss the direction? I believe the chip/IP vendors will also be interested in the clarification, so that they can invest properly moving forward.