Tensorrt: PReLU layer without channel sharing

[xonobo]

There is an NvPlugin for PReLU and its creator is defined as

TENSORRTAPI INvPlugin* createPReLUPlugin(float negSlope);

which takes a single floating value. The default PReLU layer defined in caffe does not share negSlope values over channels and each channel's negSlope value is kept in a weight parameter of the layer.

My questions:
1 - Does NvPlugin for PReLU supports different negSlope for each channel?
2 - If so how can I create plugin that parse the weights from the caffemodel file?
3 - If not what is your advice to implement default PReLU caffe layer for TRT with a smooth caffeparser?

[nfeng0105]

@xonobo

1. Current preLU plugin leverages legacy leaky relu plugin and doesn't support per-channel activation. BTW, TensorRT 5.1 has native leaky relu support and the native support for prelu will be coming in next version.
2. https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/samplePlugin/fcPlugin.h demonstrates how to create a weighted plugin layer.
3. If you want caffeparser to be able to parser your plugin layer and its parameter automatically , you can refer to priorbox plugin,
   https://github.com/NVIDIA/TensorRT/blob/master/parsers/caffe/caffeParser/caffeParser.cpp#L508
   https://github.com/NVIDIA/TensorRT/blob/master/parsers/caffe/caffeParser/caffeParser.cpp#L90
   If you have plugin implementation and its parsing function available, then it behaves like a native layer.
   BTW, parsing the plugin parameters may require to update trtproto,
   https://github.com/NVIDIA/TensorRT/blob/master/parsers/caffe/proto/trtcaffe.proto

[xonobo]

Till the next version I will use this WAR.

https://github.com/PKUZHOU/MTCNN_FaceDetection_TensorRT

Thanks for the detailed examples and quick response.