I notice LeakyReLU has been natively supported in lastest TensorRT. And my experiment showed a large performance gap between fused Conv+ReLU and currently separated Conv+LeakyReLU. So will fusion of Conv + LeakyReLU be supported in your timetable?
More fusions with GEMM's/Conv is being considered in future versions of TensorRT.
Look forward to your good news. Thank you!
Most helpful comment
More fusions with GEMM's/Conv is being considered in future versions of TensorRT.