Onnx-tensorrt: Gather not properly supported

[jinfagang]

When convert maskrcnn.onnx model, this model actually can successfully inference using onnxruntime CPU, but onnx2trt not properly parse Gather node:

onnx-tensorrt/builtin_op_importers.cpp:772 In function importGather:
[8] Assertion failed: !(data->getType() == nvinfer1::DataType::kINT32 && nbDims == 1 && inputs.at(0).is_tensor()) && "Cannot perform gather on a shape tensor!"

this assertions seems need to properly handle .

[akirasosa]

It can be reproduced by using shape in pytorch. Here is an example.

class Net(nn.Module):
    def forward(self, x):
        return x.shape

As a result, a lot of models don't work.

[jinfagang]

@akirasosa Exactly, IDK what these assertions means and indeed some of them does not really necessary or should handle some corner situations.

[akirasosa]

I used https://github.com/daquexian/onnx-simplifier and my problem was solved.

[jinfagang]

@akirasosa This lib just simplify some redundant nodes, can not applying on large models. I have tried this tool before.

So, what does the differences between converts and after? Can u take a look of inside model and what changed inside Gather?

[UJWALA-SANKH]

Hey any update on this issue? facing this exact same error

[jdr-face]

Hey any update on this issue? facing this exact same error

The suggestion above from @akirasosa to use https://github.com/daquexian/onnx-simplifier works perfectly here.

[mehdiAtCellarEye]

Hi there,
Any update on this issue? I have a similar problem with a Faster R-CNN pytorch model converted to ONNX. I do not have any problem running the model with onnxruntime-gpu (compiled with CUDA support). However, I get the following error when trying to run it using onnxruntime compiled with TensorRT support:

ShapeTensor.cpp:38: onnx2trt::ShapeTensor::ShapeTensor(int, int32_t): Assertionsize_ >= 0' failed.`

Really appreciate any help. I indeed tried the onnx-simplifier as pointed out above. But it failed without success.

[ghost]

The assertion is pretty confusing because in split:

DEFINE_BUILTIN_OP_IMPORTER(Split)

{

    const int numOutputs = node.output().size();



    nvinfer1::ITensor* tensorPtr = &convertToTensor(inputs.at(0), ctx);

    const int rank = tensorPtr->getDimensions().nbDims;

    nvinfer1::ITensor* shape = ctx->network()->addShape(*tensorPtr)->getOutput(0);



    OnnxAttrs attrs(node);

    int axis = attrs.get<int>("axis", 0);

    TRT_CHECK(convert_axis(axis, rank));



    std::vector<int> outputLengths;

    if (attrs.count("split"))

    {

        outputLengths = attrs.get<std::vector<int>>("split");

        ASSERT(static_cast<int>(outputLengths.size()) == numOutputs, ErrorCode::kINVALID_NODE);

    }



    nvinfer1::ITensor* startSliceAxis{addConstantScalar<int32_t>(ctx, 0, ::ONNX_NAMESPACE::TensorProto::INT32, nvinfer1::Dims{1, 1})->getOutput(0)};

    // sizeSliceAxis = axisLength / numOutputs

    nvinfer1::ITensor* sizeSliceAxis{ctx->network()->addElementWise(

        *gatherDimension(ctx, shape, axis, nvinfer1::Dims{1, 1}),

        *addConstantScalar(ctx, numOutputs, ::ONNX_NAMESPACE::TensorProto::INT32, nvinfer1::Dims{1, 1})->getOutput(0),

        nvinfer1::ElementWiseOperation::kDIV

    )->getOutput(0)};
   ...
}

The input tensor to addGather(enclosed in gather Dimension) is exactly a shape tensor.

[kevinch-nv]

Is anyone still having trouble with this with the latest TensorRT release?

[kevinch-nv]

Closing due to inactivity - if you are still having issues with the latest version of onnx-tensorrt feel free to open a new issue.