Onnx-tensorrt: > `OnnxParser` and `onnx2trt` - different behaviour

[felixplum]

Hi,

I was wondering about the difference between OnnxParser (called from TensorRT-interface in Python) and the onnx2trt-binary (as compiled from C++-source). While the first failed to convert my model by throwing a segfault (floating point exception)*, the latter works just fine.

It could be either that TensorRT is linking to a buggy onnx-tensorrt release (https://github.com/onnx/onnx-tensorrt/tree/c153211418a7c57ce071d9ce2a41f8d1c85a878f), or that the Python bindings of onnx-tensorrt are causing the issue.

A similar issue was encountered by @laggui in https://github.com/onnx/onnx-tensorrt/issues/180#issuecomment-523712339_

*gdb says:

"python3" received signal SIGFPE, Arithmetic exception.
0x00007fffe8de78dc in nvinfer1::ShuffleLayer::getOutputDimensions(nvinfer1::Dims const&) const () from /opt/TensorRT-6.0.1.5/lib/libnvinfer.so.6

when running parser.parse(onnx_model.read())

Edit: Invoking the current Python backend directly:

model = onnx.load( 'test.onnx')
engine = backend.prepare(model, device='CUDA:0')

fails with error

[TensorRT] ERROR: (Unnamed Layer* 0) [Convolution]: at least 4 dimensions are required for input
Traceback (most recent call last):
  File "trt_demo.py", line 21, in <module>
    engine = backend.prepare(model, device='CUDA:0')
  File "/usr/lib/python3/dist-packages/onnx_tensorrt/backend.py", line 200, in prepare
    return TensorRTBackendRep(model, device, **kwargs)
  File "/usr/lib/python3/dist-packages/onnx_tensorrt/backend.py", line 86, in __init__
    raise RuntimeError(msg)
RuntimeError: While parsing node number 1:
builtin_op_importers.cpp:695 In function importBatchNormalization:
[6] Assertion failed: scale_weights.shape == weights_shape

Cheers
Felix

[andrew-yang0722]

when I compile it, I added ONNX related options. like the below
and then the error don't occur anymore.

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_BIN_DIR=`pwd`/out -DBUILD_ONNX_PYTHON=ON -DONNX_BUILD_BENCHMARKS=ON -DONNX_BUILD_TESTS=ON

[zhangjinsong3]

I meet the same error with python. I am just converting an resnet-based classification model:

Traceback (most recent call last): File "tools/onnx2trt.py", line 40, in <module> engine = backend.prepare(model, device='CUDA:0') File "../onnx-tensorrt/onnx_tensorrt/backend.py", line 200, in prepare return TensorRTBackendRep(model, device, **kwargs) File "../onnx-tensorrt/onnx_tensorrt/backend.py", line 86, in __init__ raise RuntimeError(msg) RuntimeError: While parsing node number 1: builtin_op_importers.cpp:695 In function importBatchNormalization: [6] Assertion failed: scale_weights.shape == weights_shape

[anuar12]

I had the same error when I accidentally compile from master instead of 6.0 branch. I ran cmake with @andrew-yang0722 command (without DTRT arguments) and didn't get that error.

[LuYanFCP]

I meet the same error when I run the model of shuffleNetV2 which I had converted from pytorch model zoo .

import onnx
import onnx_tensorrt.backend as backend
import numpy as np

input_data = np.random.randn(1, 3, 244, 244).astype(np.float32)

model = onnx.load("shuffleNetV2.onnx")
engine = backend.prepare(model, device='CUDA:0')
output_data = engine.run(input_data)[0]

print(output_data)
print(output_data.shape)

error:

Traceback (most recent call last):
  File "torch_rato.py", line 9, in <module>
    engine = backend.prepare(model, device='CUDA:0')
  File "/data/users/ly/miniconda3/envs/tensorrt/lib/python3.6/site-packages/onnx_tensorrt-0.1.0-py3.6-linux-x86_64.egg/onnx_tensorrt/backend.py", line 200, in prepare
    return TensorRTBackendRep(model, device, **kwargs)
  File "/data/users/ly/miniconda3/envs/tensorrt/lib/python3.6/site-packages/onnx_tensorrt-0.1.0-py3.6-linux-x86_64.egg/onnx_tensorrt/backend.py", line 86, in __init__
    raise RuntimeError(msg)
RuntimeError: While parsing node number 1:
builtin_op_importers.cpp:695 In function importBatchNormalization:
[6] Assertion failed: scale_weights.shape == weights_shape

[phamdat09]

Hello!!
Anyone can fix it ?

[121786404]

Anyone can fix it ?

[kevinch-nv]

We've recently made some upgrades to the onnx-tensorrt Python backend that should streamline it's installation.

As for the onnx2trt executable, we plan on deprecating it in a future release. I would recommend using our command line tool trtexec to quickly run and benchmark ONNX models.

If you are still having issues with the latest commit regarding the python backend or trtexec, feel free to open a new issue.