Tensorrt: (Unnamed Layer* 4) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor

[zys1994]

Description

tensorrt7 support prelu in onnx. but i use onnx parser for pRelu have problem

code

int verbosity = (int) nvinfer1::ILogger::Severity::kWARNING;

    IBuilder* builder = createInferBuilder(gLogger);
    const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
    nvinfer1::INetworkDefinition* network = builder->createNetworkV2(explicitBatch);

    nvonnxparser::IParser* parser =nvonnxparser::createParser(*network, gLogger);
    if (!parser->parseFromFile(modelFile.c_str(), verbosity)) //解析onnx文件，并填充网络
    {
        std::string msg("failed to parse onnx file");
        gLogger.log(nvinfer1::ILogger::Severity::kERROR, msg.c_str());
        exit(EXIT_FAILURE);
    }

Error

Input filename:   /home/bill/E/study/ncnn/ncnn_lnet/model/lnet106_112/landmark_112.onnx
ONNX IR version:  0.0.3
Opset version:    8
Producer name:    
Producer version: 
Domain:           
Model version:    0
Doc string:  
[01/03/2020-19:40:59] [E] [TRT] (Unnamed Layer* 4) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[01/03/2020-19:40:59] [E] [TRT] (Unnamed Layer* 4) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
[01/03/2020-19:40:59] [E] [TRT] (Unnamed Layer* 4) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor
While parsing node number 3 [Conv]:
ERROR: builtin_op_importers.cpp:461 In function importConv:
[8] Assertion failed: nbSpatialDims == kernel_weights.shape.nbDims - 2

Environment

TensorRT Version: 7.0.0.11
GPU Type: Quadro P2000
Nvidia Driver Version: NVIDIA-SMI 390.87
CUDA Version:cuda 9.0
CUDNN Version: cudnn 7.6.3
Operating System + Version: ubuntu

[rmccorm4]

Hi @zys1994,

You need to make sure your PReLU op (the slope) follows the rules here for unidirectional broadcasting:

1. https://github.com/onnx/onnx/blob/master/docs/Operators.md#prelu
2. https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md#unidirectional-broadcasting

[zys1994]

PRelu takes input data (Tensor) and slope tensor as input, and produces one output data (Tensor) where the function f(x) = slope * x for x < 0, f(x) = x for x >= 0., is applied to the data tensor elementwise. This operator supports unidirectional broadcasting (tensor slope should be unidirectional broadcastable to input tensor X); for more details please check the doc.

Version
This version of the operator has been available since version 9 of the default ONNX operator set.

my onnx version is ver8, Does it mean that ver8 cause the problem and i should upgrade onnx to version>9. Thanks

[rmccorm4]

It might be, but I don't think so. There's also a PReLU op from ONNX version 7: https://github.com/onnx/onnx/blob/master/docs/Changelog.md#PRelu-7

I think the only difference from looking at it is Version 9 added support for more input/output types.

I'm assuming the error listed is right, and that your slope tensor shape is incorrect/unsupported.

[zys1994]

my onnx model convert from mxnet(1.5.0) to onnx(1.3.0). the net is listed like that,

tumblr post codes

Can you find that hou to solve it?

[RunningLeon]

my onnx model convert from mxnet(1.5.0) to onnx(1.3.0). the net is listed like that,

tumblr post codes

Can you find that hou to solve it?

Have you solved this problem?

[RunningLeon]

Hi @zys1994,

You need to make sure your PReLU op (the slope) follows the rules here for unidirectional broadcasting:

1. https://github.com/onnx/onnx/blob/master/docs/Operators.md#prelu
2. https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md#unidirectional-broadcasting
   Hi,
   I met same problem when try to convert mxnet->onnx->tensorRT
   Originally, prelu's slope param has shape of (C,) and the output info is :
   [TensorRT] ERROR: (Unnamed Layer* 11) [Parametric ReLU]: slope tensor must be unidirectional broadcastable to input tensor ERROR: Failed to parse the ONNX file. In node 6 (scaleHelper): UNSUPPORTED_NODE: Assertion failed: dims.nbDims == 4 || dims.nbDims == 5
   When I mannually reshape the slope to shape of (C,1,1), the output error info is like this:
   [TensorRT] WARNING: Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.
   [TensorRT] ERROR: Layer (Unnamed Layer* 245) [Scale] failed validation
   [TensorRT] ERROR: Network validation failed.

Pls. kindly help me with this if possible.
Many thanks in advance.
BR

[cloudhan]

If you are using mxnet to onnx, you need to monkey patch the conversion script with unsqueezed gamma tensor.
In mxnet prelu's gamma is stored as of shape [num_channel], whereas onnx unidirectional-broadcastable requires it to be of shape [1, num_channel, 1, 1].

[RunningLeon]

If you are using mxnet to onnx, you need to monkey patch the conversion script with unsqueezed gamma tensor.
In mxnet prelu's gamma is stored as of shape [num_channel], whereas onnx unidirectional-broadcastable requires it to be of shape [1, num_channel, 1, 1].

Hi,
Thanks, I reshaped slope of PReLU to [1, num_channel, 1, 1] and it worked.
BR

[iobtl]

Hi @RunningLeon, could you share how did you manage to reshape the slope of PReLU using MXNet? Was it using the mx.Symbol API as such:
mx.sym.LeakyReLU(data=data, ...)

[RunningLeon]

Hi @RunningLeon, could you share how did you manage to reshape the slope of PReLU using MXNet? Was it using the mx.Symbol API as such:
mx.sym.LeakyReLU(data=data, ...)

First loaded mxnet checkpoint and reshape prelu param in arg_param and than save the new checkpoint and finally export it to onnx.
something like this:
`
model_dir = 'xxx/'
model_path = model_dir + 'model'
num_epoch = 1
sym, arg_params, aux_params = mx.model.load_checkpoint(model_path, num_epoch)
new_arg_params = {}
for k, v in arg_params.items():
if 'relu' in k:
v = v.reshape(1, -1, 1, 1)
new_arg_params[k] = v

new_model_path = model_path + '_test'
mx.model.save_checkpoint(save_prefix, num_epoch, sym, new_arg_params, aux_params)

sym_path = new_model_path + '-symbol.json'
params_path = new_model_path + '-{:04d}.params'.format(num_epoch)
onnx_out_path = model_path + "_test.onnx"
onnx_mxnet.export_model(sym_path, params_path, [(1, 3, 112, 112)], np.float32, onnx_out_path)

Load onnx model

model_proto = onnx.load_model(onnx_out_path)

Check if converted ONNX protobuf is valid

checker.check_graph(model_proto.graph)
`
BR

[iobtl]

@RunningLeon Thank you!!!!

[jiagh2010]

export-model.py in create_onnx_graph_proto IndexError: list index out of range.

[rajeevsrao]

Thanks @RunningLeon. @zys1994 please let us know if this issue can be closed now.

[ttyio]

I will close this since no response for more than 3 weeks, please reopen if you still have question, thanks!