Mlpack: Addition of Depth-wise Separable Convolution (Separable Convolution) Layer.

[kartikdutt18]

Hi everyone, I was working on implementing mobile-net for when I noticed that I would have to implement depth-wise convolution layer so I thought rather than implementing it only for mobile-net, Adding it to ann/layers might be better for other architectures such as efficient net etc.
Depth-Wise convolution layers have the following advantages:

1. Grouped Convolution require less RAM and time and since mlpack is CPU only for now, it might be
   better to depth-wise convolution for projects (especially on devices like raspberry pi).
2. Each kernel filter convolves on all the feature maps obtained on previous layer, resulting in lots of
   convolutions, some of which may be redundant.
3. Many SOTA networks use this layer and this layer is already implemented in popular libraries such
   as keras(depthwise), pytorch (groups), tensprflow (seperable_conv).
   I wanted to pursue this topic so that I could use it mobilenet directly rather implementing one there, Would it be okay if I pursue this issue.

[kartikdutt18]

Hi @zoq , would this be okay or should I implement it in mobile-net locally?

[zoq]

Absolutely, let me know if you need any help.

[kartikdutt18]

Thanks, I will start working on it.

[kartikdutt18]

Hi @zoq,Could you help me this https://github.com/mlpack/mlpack/pull/2172#discussion_r375355745 ,
Sorry that I keep disturbing you.
Thanks.

[mlpack-bot[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! :+1: