Onnx-tensorrt: Error in Transpose while generating trt engine

[CNSaintNeo]

If I convert a onnx model including Transpose layer to a trt plan, trt_builder.build_cuda_engine(trt_network) return NULL.
But if I discard the Transpose layer, the trt engine can be built successfully.

How can i get trough this problem？ Thanks！

[yinghai]

@benbarsdell Could you take a look?

[CNSaintNeo]

The error is:
364: Transpose -> []
terminate called after throwing an instance of 'std::out_of_range'
what(): Attribute not found: shape

My onnx file is:
%364 = Transposeperm = [0, 2, 3, 1]
The original order of Tenser is [N, C, H, W]=(1, 32, 64, 48), the transposed order is [N, H, W, C]=(1, 48, 32, 64),

[yinghai]

@CNSaintNeo Do you have a minimal reproducible onnx model that we can take a look at?

[Michalaq]

I also encountered this problem.
Maybe not minimal, but a simple script to reproduce that:

from mnist import Net
import torch
import torch.onnx
from torch import nn
from torch.nn import functional as F
import tensorrt as trt
from tensorrt.parsers import onnxparser

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.softmax(x, dim=1)

def get_parser(filename):
    apex = onnxparser.create_onnxconfig()
    apex.set_model_file_name(filename)
    apex.set_model_dtype(trt.infer.DataType.FLOAT)
    apex.set_print_layer_info(True)
    apex.set_verbosity_level(3)
    trt_parser = onnxparser.create_onnxparser(apex)
    data_type = apex.get_model_dtype()
    onnx_filename = apex.get_model_file_name()
    trt_parser.parse(onnx_filename, data_type)
    return trt_parser

dummy_input = torch.ones(10, 1, 28, 28).to('cuda')
q = Net()
q.cuda()

# make sure net is runnable
q(dummy_input)

torch.onnx.export(q, dummy_input, "mnist.onnx", verbose=True)


trt_parser = get_parser('./mnist.onnx')
trt_parser.convert_to_trtnetwork()

The network is from pytorch's MNIST example. The only thing I changed was to change F.log_softmax to F.softmax, because apparently onnx does not support LogSoftmax.

Obviously this can be probably furtherly striped down to the actual source of error, but unfortunately I don't have much time now. (Will try to look into later, but if anyone helps with that I would appreciate it).

I get the same error about attribute "shape" not found.

...
14: Relu -> [20,4,4]
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: shape

Tested with:

ArchLinux, kernel 4.17
pytorch 0.4.1
cuda 9.2
tensorrt 4.0.1.6 (installed from tar)
onnx installed from source from master, commit 69674ffe67f3ad3085185a4ad5784535a38a96c6
onnx-tensorrt installed from source

Btw, it would be nice to mention it needs gcc 5 to be compiled and swig to build python lib.
Also, since by default TensorRT "installed" from tar (so just decompressed) does not land in any specific directory, it would be nice to be able to specify path to TensorRT in some nicer way than manually editing setup.py to add a line "-I/path/to/tensorrt/include".

[yurymalkov]

@Michalaq This can be solved in the sample by replacing x = x.view(-1, 320) with x = x.view(int(x.shape[0]), 320)

[shocho3858]

If I convert a onnx model including Transpose layer to a trt plan, trt_builder.build_cuda_engine(trt_network) return NULL.
But if I discard the Transpose layer, the trt engine can be built successfully.

How can i get trough this problem？ Thanks！

Have you solved this? I have the same problem in transpose operation.

[thelongestmovie]

I have the same problem too.But if I use TensorRT 5.It can be solved

[Jongchan]

@Michalaq This can be solved in the sample by replacing x = x.view(-1, 320) with x = x.view(int(x.shape[0]), 320)

Problem solved in TensorRT 4.0.1 :]
Thanks!

[kevinch-nv]

Closing this issue as it looks like it's been solved. If anyone else is having issues with Transpose, please feel free to file a new issue.