Onnx-tensorrt: tensorrt7.0 Assertion failed: scales_input.is_weights()

[zhouyingchaoAI]

tensorrt 7.0 cuda10.2
While parsing node number 253 [Upsample -> "692"]:
ERROR: /opt/onnx-tensorrt/builtin_op_importers.cpp:3252 In function importUpsample:
[8] Assertion failed: scales_input.is_weights()

[zlwzlwzlw]

the same fault

[kaishian]

We encountered the same problem here.
TensorRT: 7.0
CUDA: 10.2
ONNX: 1.6.0
PyTorch: 1.4.0a0

[1LOVESJohnny]

Hi, I met totally the same problem. Have you guys solved it?

[TrojanXu]

Give it a try: (for NCHW)

To define upsample, instead of using scale_factor, using pre-calculated size

def upsample(x, scale):
sh = torch.tensor(x.shape)
return F.interpolate(x, size=(int(sh[2]scale), int(sh[3]scale)), mode='bilinear')

export the model using opset 10 for TRT7
torch.onnx.export(model, inputs, filename, verbose=True, opset_version=10)

[fpreiswerk]

I can confirm that @TrojanXu's approach works (Cuda 10.2, TensorRT 7.0, Torch 1.5). The new shape must be provided in an explicit manner instead of a scale factor. The reason is that when using a scale factor, the resulting graph dynamically calculates the output shape, which is incompatible with TensorRT (at least until today).

Some more context on the issue can be found here: https://github.com/pytorch/pytorch/issues/27376

[xpngzhng]

@TrojanXu hi
I follow your advice, but I still encounter tensorrt error

settings:

model conversion is in ubuntu18.04
pytorch 1.5
onnx build from source version 1.7

tensorrt inference is in windows10
tensorrt 7.0.0.11
cuda 10.2

here's the log

Input filename:   hrnet_w18_dist_4gpus_38r0_65d02_epoch_120_onnx1.6_opset10_bilinear_no_align.onnx
ONNX IR version:  0.0.6
Opset version:    10
Producer name:    pytorch
Producer version: 1.5
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:222: One or more weights outside the range of INT32 was clamped
While parsing node number 131 [Resize]:
ERROR: builtin_op_importers.cpp:2412 In function importResize:
[8] Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"

using the netron to check node 131, I see

all the interpolate in network definition use size instead of scale

BUT there is indeed some onnx resize node with well defined scale

UPDATE:
setting all the size of interpolate to be CONSTANTS solves the problem, really hard work

[DJMeng]

all the interpolate in network definition use size instead of scale

What is mean about "all the interpolate in network definition use size instead of scale" ?

My onnx is like this:

In my pytorch export onnx code,my net-work use:
y = F.interpolate(y, size=sources[2].size()[2:], mode='bilinear', align_corners=False)

when i convert onnx to trt, it have errors:

While parsing node number 63 [Resize]: ERROR: ModelImporter.cpp:124 In function parseGraph: [5] Assertion failed: ctx->tensors().count(inputName)

Do you meet this errors? If you are available, please help me, Thanks very much.

[xpngzhng]

@DJMeng
according to my experience, the size argument of F.interpolate should be constant, e.g. (1024, 1024)

[DJMeng]

@DJMeng
according to my experience, the size argument of F.interpolate should be constant, e.g. (1024, 1024)

Thanks, I have known it about the size replace. Do you succeed in convertting the onnx to trt?

[xpngzhng]

@DJMeng yes, you can see my last last post, the relevant software package versions are listed. But the result are not consistent, see this https://github.com/onnx/onnx-tensorrt/issues/457

[DJMeng]

@DJMeng yes, you can see my last last post, the relevant software package versions are listed. But the result are not consistent, see this #457
Yerp, I can use opset10 to solved the fails of exporting trt, But I still meet this problem: ' the python's infer result is different with the trt infer result '. It bothers me two days.

[donrax]

If anyone is experiencing similar issues concerning the resize layer and dynamic or shape inputs when loading a pytorch exported onnx model with tensorrt. We were able to bypass the issue by:

1) Setting the explicit size in F.interpolate (as mentioned in https://github.com/onnx/onnx-tensorrt/issues/361#issuecomment-625606909)

xh, xw = int(x.shape[-2]*2), int(x.shape[-1]*2)
x = F.interpolate(x, size=(xh, xw))

2) Removing the dynamic_axes parameter in torch.onnx.export and tracing a fixed size input tensor

Pytorch 1.5.1
Tensorrt 7.0
Opset 10

Might help with #423 #414 #302 #275

[franciszq]

@TrojanXu hi
I follow your advice, but I still encounter tensorrt error

settings:

model conversion is in ubuntu18.04
pytorch 1.5
onnx build from source version 1.7

tensorrt inference is in windows10
tensorrt 7.0.0.11
cuda 10.2

here's the log

Input filename:   hrnet_w18_dist_4gpus_38r0_65d02_epoch_120_onnx1.6_opset10_bilinear_no_align.onnx
ONNX IR version:  0.0.6
Opset version:    10
Producer name:    pytorch
Producer version: 1.5
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2020-10:58:45] [W] [TRT] onnx2trt_utils.cpp:222: One or more weights outside the range of INT32 was clamped
While parsing node number 131 [Resize]:
ERROR: builtin_op_importers.cpp:2412 In function importResize:
[8] Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"

using the netron to check node 131, I see

all the interpolate in network definition use size instead of scale

BUT there is indeed some onnx resize node with well defined scale

UPDATE:
setting all the size of interpolate to be CONSTANTS solves the problem, really hard work

how to set to be CONSTANTS ? I only have the models of nearest | linear | bilinear | bicubic | trilinear

[kevinch-nv]

I recommend using TRT 7.2 and opset 11 for exporting upsample and resizes