Onnx-tensorrt: Assertion failed: tensors.count(input_name) error when converting onnx to tensorrt

[tairen99]

Hi, all,
I am trying to convert a trained tiny yolov2 model into tensorrt format.
I was able to transform the tiny_yolov2.weights into tiny_yolov2.onnx using onnxmltools. But when I tried to use onnx2trt, I got this error:

---

Input filename: tiny_yolov2.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: OnnxMLTools
Producer version: 1.6.0
Domain: onnxconverter-common
Model version: 0

Doc string:

Parsing model
While parsing node number 30 [Mul -> "scalerPreprocessor_scaled"]:
ERROR: /home/cisco/onnx-tensorrt-5.1/ModelImporter.cpp:552 In function importModel:

[5] Assertion failed: tensors.count(input_name)

I am using the ubuntu 18.04 LTS; Tensorrt 5.1.6; and onnx-tensort 5.1 version.
Thank you!

[JimKlingshirn]

I had the same problem, using PyTorch 1.3. Using PyTorch 1.2 with torchvision 0.4 works better for me.

[rmccorm4]

Hi,

I think this error comes from the ONNX parser having an issue parsing your graph, specifically that it found a node with no input I believe.

There are a few things you can try for this:

1. It seems that PyTorch 1.3 is a little more up to date than the current parsers (opset 11), so that may be the root cause for now. You can try downgrading to 1.2 just for exporting your Pytorch to ONNX, as mentioned above.

2. You can try inspecting your graph with the ONNX python library to see if it gives you any hints to the issue:

import onnx
onnx.checker.check_model(onnx_model)

1. You can visually inspect your graph using something like https://lutzroeder.github.io/netron/ to see if anything looks wrong with the graph.

2. One user even tried changing the parser code, so there may be some helpful things here, but I don't think that is generally necessary: https://github.com/onnx/onnx-tensorrt/issues/199

[j-o-d-o]

Get the same error with a Tensorflow MobileSSD net. The check_model() said everything is ok : /
I don't see any of my nodes not having any input when I debug my graph.

[j-o-d-o]

Updating to TensorRT 7.0.0 somehow fixed it

[PythonImageDeveloper]

I want to convert ssd_mobilenet_v3_small_coco_2019_08_14.onnx model to tensorrt, and I get the same error with jetson nano, How I do to upgrade TensorRT6 to 7?

[rmccorm4]

@PythonImageDeveloper TensorRT 7 hasn't been released yet for Jetpack (Jetson). Sorry, I can't say anything about upcoming roadmaps.

[twmht]

pytorch==1.2.0
onnx==1.6.0
TensorRT==7.0.0.11

should be a workaround.

It seems that there is a dummy constant of resize layer if you use pytorch 1.3 or newer.

[ankitdhall]

TensorRT-7 with the latest Pytorch+torchvision version gives the same error

[maaft]

Can confirm, still gives same error.

[Bisgates]

Set opset_version to 10 somehow solved the problem : |

[ankitdhall]

@Bisgates can you post your system specs/library versions/CUDA version?

[tom-garford]

I am also facing this problem on TRT 6.0, however, have had success with TRT 7.0 and making this change https://github.com/onnx/onnx-tensorrt/pull/369/files.

The main issue seems to be a interpolation layer at the start of my network (F.interpolate) but have noticed differencess ONNX opsets:

Opset 11 changes nn.Upsample and F.interpolate layers to "Resize layers" with a constant input as well as previous layers:

Opset 10:

and Opset 9:

Despite warnings when converting ONNX, I get success with opset 9 and opset 10 providing "size=" is used rather than a scale factor for nn.Upsample or F.interpolate layers. However, I still get count(input_name) error with opset 11. All done with pytorch 1.3.

As mentioned, my main aim is to run on Jetson devices which only support TRT 6.0 so my engine file created with TRT 7.0 still doesn't work.

[tom-garford]

Anyone who has run out of ideas try with torch==1.0.1 torchvision==0.22 instead. This worked for me when previously got count(input_name) errors whatever opsets I used when using pytorch 1.3 to create engine files for TRT 6.0. This worked even with F.interpolate and Upsample layers (yes, I know, I should update them) in the same model

[guischmitd]

I'm having trouble with this conversion as well. My goal is to run models on the jetson.
Error log:

Loading ONNX file from path resnet50.onnx...
Beginning ONNX file parsing
[TensorRT] VERBOSE: input_1_1:01_permuted:Transpose -> (3, 224, 224)
ERROR: Failed to parse the ONNX file.
In node 1 (importModel): INVALID_GRAPH: Assertion failed: tensors.count(input_name)

• .onnx model created with keras2onnx with opset 10 (for TRT 6.0 compatibility)
• checker.check_model() doesn't give an error nor does it output anything (I'm assuming that's a good sign)
• Tried removing every custom code from my model, I'm simply trying to port the default tf.keras.applications.ResNet50 model using onnx as an intermediate representation (tf-trt has been a pain to work with)

Desktop Environment (model training and keras2onnx conversion):

• Ubuntu 18.04
• Tensorflow 1.15.2
• keras2onnx 1.6.5
• onnx 1.6.0

Jetson environment (onnx to tensorrt conversion)

• JetPack 4.3
• TRT 6.0.1.10

[tom-garford]

Hi @guischmitd I would first definitely try with opset 9 if possible. Also, I would use netron to visualise my model and make sure everything looks good. Also, what's the permute/ transposing referencing? Do you have permute layers? I would double check if they are compatible in TRT.

Tom.

[guischmitd]

Hey @tom-garford thanks for the reply! I've tried using opset 9 as well and everything seems ok on netron (regarding inputs and outputs, disconnected layers and such). I've noticed that there's a transpose layer at the start of my network, which I assume derives from the channels last standard of Keras networks.

I've also crossed out installation problems since I can run the darknet -> onnx -> trt example from this post.
I'll try changing the input to NCHW on the keras model to see if it makes a difference. Will report back with (hopefully good) results!

[tom-garford]

Hi @guischmitd

Good to hear! Yes, I would try and scrub out that transpose layer and see what happens. I know from bitter experience the input must be NCHW not NHWC (RRRRGGGBBB, not RGBRGBRGB, etc.) for tensorRT anyway. I would wager you at least get a more informative error message (if not solve the problem) by removing the transpose layer.

[guischmitd]

Ok, here's the latest update:
I was able to change the keras standard to channels_first using tensorflow.keras.backend.set_image_data_format('channels_first') to generate a new graph:

You can see the transpose layer is gone here. Unfortunately for some reason the flatten layer after my convolution blocks creates another transpose layer:

I've tried time and again with different models from tensorflow.keras.applications (Mobilenet, InceptioV3, ResNet50) but none of these can be parsed by the tensorrt.ONNXParser.

I could finally run my engine builder on the jetson after switching to opset 7 on keras2onnx.convert_keras(path_to_h5_model, target_opset=7). I've yet to test inference using the generated .trt engine. Once again, I'll post my results here for future reference.

Thanks for the help so far!

[tom-garford]

Hi @guischmitd great news!

Glad you finally got it going. As you saw from my attempts, it takes a bit of experimentation! Thanks for posting your results.

Tom.

[DJMeng]

TensorRT-7 with the latest Pytorch+torchvision version gives the same error

I meet the same errors. Have you solved it ? Thanks.

[Darshcg]

Hi,
I am Converting my Custom Pytorch Model to TRT Engine file. I am able to convert the model to Onnx, but I am facing an Issue while converting from Onnx to the Engine file.
I am getting the error as:
AttributeError: ‘NoneType’ object has no attribute ‘serialize’.
I am giving the correct shape as an input to the build engine function.
And for the Resnet50 model, I am able to convert successfully but for my custom model, I am facing the above issue.
Please suggest if there is any specific reason behind that.

I tried with trtexec command, I am getting the error as attached below:

While parsing node number 195 [Resize]:
ERROR: ModelImporter.cpp:124 In function parseGraph:
[5] Assertion failed: ctx->tensors().count(inputName)
[09/25/2020-11:02:52] [E] Failed to parse onnx file
[09/25/2020-11:02:52] [E] Parsing model failed
[09/25/2020-11:02:52] [E] Engine creation failed
[09/25/2020-11:02:52] [E] Engine set up failed

[kevinch-nv]

Thanks for those in the thread that provided help. This issue should be fixed in later TensorRT releases, and I would recommend anyone who is facing this issue to try the latest TensorRT version to parse their model.

If you are still hitting this issue, please open a new one specific to your model. Thanks!

[wwqstevenash]

I have the same problem to convert interpolate from torch to onnx to tensorrt.
I sovle it with only change the param "opset_version" in the function torch.onnx.export.
pytorch==1.4.0
onnx==0.7.0
opset_version=10

The differnt with two param "opset_version=10" and "opset_version=11" in the onnx used netron app.
opset_version=10

opset_version=11

[jovialio]

If it's issue relating to F.Interpolate, this is what i raised and the work around i found to convert from Torch->Onnx->TensorRT.

https://github.com/onnx/onnx-tensorrt/issues/615

[hoangmt]

Ok, here's the latest update:
I was able to change the keras standard to channels_first using
I could finally run my engine builder on the jetson after switching to opset 7 on keras2onnx.convert_keras(path_to_h5_model, target_opset=7). I've yet to test inference using the generated .trt engine. Once again, I'll post my results here for future reference.

Thanks for the help so far!

The opset value seems to be tricky to me. Can anybody shine some light on the meaning of the values and how to set the value?

One more simple example to play it.

import tensorflow as tf
import onnx_graphsurgeon as gs
import onnx
import numpy as np
saved_model_path='/usr/tmp/model1'
from tensorflow.keras.layers import Input, Conv2D, Dense, Flatten
from tensorflow.keras import Model

input_shape = (30, 30,3)
inputs = Input(shape=input_shape,name='input')
x = Conv2D(2, 3, activation='relu')(inputs)
out= Dense(400, activation='relu',name='output')(x)
model = Model(inputs=inputs, outputs=[out])

tf.saved_model.save(model1,saved_model_path)
!python3 -m tf2onnx.convert --saved-model {saved_model_path} --opset 7 --output {saved_model_path+'.onnx'}
!/usr/src/tensorrt/bin/trtexec --onnx={saved_model_path+'.onnx'} --saveEngine=model.trt

Change opset value to see the error.