Onnx-tensorrt: [Question] onnx2trt vs trtexec --onnx

[austinmw]

Hi, what's the difference between onnx2trt and trtexec --onnx? Are they the same thing? If not, is one preferred over the other? Also if different, and if I need to add a onnx parser plugin for a custom layer, should I use one over the other?

Also, I don't see an engine.serialize() option for onnx_tensorrt.tensorrt_engine.Engine, compared to tensorrt.ICudaEngine. Is there a major difference between Engine and ICudaEngine, and is there a way to save Engine to trt file?

[casbolwerk]

Does anyone have info on this? Especially the second question for me personally

[buganini]

Support of quantization is anoher part of question.

[kevinch-nv]

I would recommend using trtexec over onnx2trt as the CLI interface for converting ONNX models into TensorRT engines since it is more actively maintained.

onnx2trt hasn't been updated in a while, and we are planning to deprecate it in favor of trtexec soon.

[Sayyam-Jain]

I would recommend using trtexec over onnx2trt as the CLI interface for converting ONNX models into TensorRT engines since it is more actively maintained.

onnx2trt hasn't been updated in a while, and we are planning to deprecate it in favor of trtexec soon.

Can you please tell me how to add multiple optimisation profiles using trtexec. I can't seem to find any example anywhere. I know it is possible to do using Python and C++ API but I've no idea about trtexec.

Thanks

[kevinch-nv]

@Sayyam-Jain please see the last example here: https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec#example-4-running-an-onnx-model-with-full-dimensions-and-dynamic-shapes

[Sayyam-Jain]

@kevinch-nv Thank you but I meant Optimisation Profiles where I can set multiple min, opt and max sizes for different optimisation profiles.

Eg. like this: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#opt_profiles
but only using trtexec. Using trtexec, I can set maximum only 1 profile (min, opt & max) but with Python API you can add multiple such profiles