Tensorrt: how to resize a shape [256,1,24,24] to [1,256,24,24]?

[root12321]

[rmccorm4]

Hi @root12321 ,

It looks like you can use the IShuffleLayer to reshape, and leave the transpose operations as defaults (identity operation): https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#tensorrt.IShuffleLayer

Alternatively, TensorRT supports the Reshape layer from most popular frameworks as mentioned in the Support matrix: https://docs.nvidia.com/deeplearning/sdk/tensorrt-support-matrix/index.html

So you also should be able to add the Reshape to your original model before converting to TensorRT.

[root12321]

@rmccorm4 but i dont know how to use the API IShuffleLayer in python ,it seems has a add_shuffle function in INetworkDefinition ,but add_shuffle function just has one input parameters, i dont konw how to reshape a tensor by using add_shuffle function ,can you help me to solve it? thank you

[root12321]

cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0)),and the result is not changed
before use network.add_shuffle (1, 256, 4, 4) after use network.add_shuffle (1, 256, 4, 4)
how can i do it @rmccorm4

[rmccorm4]

I'm not quite sure how to add intermediate layers to an already parsed network. I believe it's possible with graphsurgeon, but I think you'll have a much easier time if you adjust your original model in Tensorflow/Pytorch/etc. before parsing it.

[root12321]

in fact ,i am using pytorch->trt, and not use pytorch->onnx->trt，i am just load pytorch model weights,and convert it to trt, just like
python conv_search_w = weights['rpn_head.cls.conv_search.0.weight'].numpy() conv_search_cls = network.add_convolution( input=batchNormLayer_5.get_output(0), num_output_maps=256, kernel_shape=(3, 3), kernel=conv_search_w, bias=trt.Weights() ) conv_search_cls.stride = (1, 1)
@rmccorm4

[rmccorm4]

I think it would look something like this to swap the first 2 dimensions, according to the docs:

cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0))
# Original indices are [0, 1, 2, 3], so [1, 0, 2, 3] should swap the first two dimensions
cls_kernel_reshape.first_transpose = trt.Permutation([1, 0, 2, 3])

https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#ishufflelayer

[root12321]

thank you very much , and i succeed to use the code

cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0))
cls_kernel_reshape.first_transpose = trt.Permutation([1, 0, 2, 3])

finish it .

[root12321]

@rmccorm4

[root12321]

here has a question ,how can i add a dimension in a tensor. for example ,i want to convert [256,1,4,4] to [256,1,1,4,4],how can i do it ? thank you @rmccorm4

[rmccorm4]

For unsqueezing a single dimension with fixed shapes, I think you can just do this:

cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0))
cls_kernel_reshape.reshape_dims = [256, 1, 1, 4, 4]

For the general case, you can combine shape/gather/concat, something like this:

shape = network.add_shape(inp)
dim0 = network.add_gather(shape, [0])
dim1 = network.add_gather(shape, [1])
dim2 = network.add_gather(shape, [2])
dim3 = network.add_gather(shape, [3])
new_shape = network.add_concat([dim0, dim1, network.add_constant([1]), dim2, dim3])
unsqueezed = network.add_shuffle(inp, new_shape)

[root12321]

thank you ,i succeed to use the code
```
cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0))
cls_kernel_reshape.reshape_dims = [256, 1, 1, 4, 4]

to convert the tensor [256,1,4,4] to [256,1,1,4,4],
when i used your code 

cls_kernel_reshape=network.add_shuffle(input=cls_kernel_relu.get_output(0))
cls_kernel_reshape.reshape_dims = trt.Permutation([256, 1, -1, 4, 4])
```

the result is [256, 1, -1, 4, 4, 4, 32676, -1128018999], i think when we use reshape_dims method， we can not use trt.Permutation for this way ,may me the official documents explain about IShuffleLayer->reshape_dims have some bug. @rmccorm4

[rmccorm4]

Thanks for the catch @root12321, I'll ask to update the docs on the expected type for reshape_dims.

Closing - Reopen if you are still having a problem.