Tensorrt: How to extract slice from tensor?

[lapolonio]

Description

I'm trying to implement the pooling output of the bert model.

I want to extract a slice from a tensor. then use that an input to a fully_connected layer.

add_slice returns an ISliceLayer which is incompatible with add_fully_connected

I can't find a way to extract weights from a single dimension of a tensor. the other idea i had was extract weights and add as a constant to network.

Environment

TensorRT Version:
GPU Type:
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Steps To Reproduce

[rmccorm4]

Hi @lapolonio,

Did you try using get_output() to get the Tensor from the Layer? add_fully_connected() expects the input to be an ITensor, not a Layer.

I haven't tried this, but might look something like:

slice_layer = network.add_slice_layer(...)
fc_layer = network.add_fully_connected(input=slice_layer.get_output(0), ...)

[lapolonio]

Thanks @rmccorm4

[churinga]

@lapolonio We have experienced similar issue. ISliceLayer cannot take -1 as shape value, so we have to know the static shape of all dimensions. However, the first shape dim is batch_size, which could differ. Actually, bert_out's tensor shape has -1 as batch_size, and it's not acceptable by the slice layer. Any idea how to fix this?

[Hoogck]

@lapolonio We have experienced similar issue. ISliceLayer cannot take -1 as shape value, so we have to know the static shape of all dimensions. However, the first shape dim is batch_size, which could differ. Actually, bert_out's tensor shape has -1 as batch_size, and it's not acceptable by the slice layer. Any idea how to fix this?

how and have you solved this problem?

[lapolonio]

@lapolonio We have experienced similar issue. ISliceLayer cannot take -1 as shape value, so we have to know the static shape of all dimensions. However, the first shape dim is batch_size, which could differ. Actually, bert_out's tensor shape has -1 as batch_size, and it's not acceptable by the slice layer. Any idea how to fix this?

how and have you solved this problem?

I don't think I did. One way might be to reshape the tensor first.