Caffe: Spatial Transformer Layer

[sontran]

This layer seems to help fine grain localization. Link to paper by Max Jaderberg et al
http://arxiv.org/abs/1506.02025
Torch implementation is here
https://github.com/qassemoquab/stnbhwd
Theano/Lasagne implementation/doc is here
https://lasagne.readthedocs.org/en/latest/modules/layers/special.html#lasagne.layers.TransformerLayer

[xygorn]

I am working on a initial version of this with an affine transformation and bilinear sampling kernel. In my initial design, I am making a layer for the grid generator and resampler together, and the localization net can be built separately for flexibility.
I will set up a pull request as soon as I port it over to the current master.

[kevinlin311tw]

I am also interested in Spatial Transformer Layer (SPL). Does it possible to embed SPL in Alexnet?

[ducha-aiki]

Take a look at https://github.com/happynear/caffe-windows/blob/master/src/caffe/layers/transformer_layer.cpp

[kevinlin311tw]

@ducha-aiki Thank you so much. I will take a look at his code. Did you try his transformer layer? Because my OS is linux, I cannot simply compile his caffe.

[ducha-aiki]

@kevinlin311tw not yet. I suppose, you could just copy-paste transform_layer.cpp/cu, entry from caffe.proto and from header to your build.

[n3011]

@sergeyk are you guys planning to include spatial transformer network with caffe?

[futurely]

@XiaoxiaoGuo implemented the spatial transformer layer too.
https://github.com/XiaoxiaoGuo/caffe-stn/blob/master/include/caffe/custom_layers.hpp
https://github.com/XiaoxiaoGuo/caffe-stn/blob/master/src/caffe/layers/spatial_transformer_layer.cpp
https://github.com/XiaoxiaoGuo/caffe-stn/blob/master/src/caffe/layers/spatial_transformer_layer.cu
https://github.com/XiaoxiaoGuo/caffe-stn/blob/master/src/caffe/test/test_spatial_transformer_layer.cpp

[siavashk]

With @XiaoxiaoGuo implementation, is it possible to perturb the transformation parameters (\theta) in a random manner during training? Similar to how the dropout layer turns some neurons off randomly.

If this is possible, you can generate spatial perturbations of data during the learning phase. This might be interesting for some people including myself.

[futurely]

Here is another implementation including complete examples. https://github.com/daerduoCarey/SpatialTransformerLayer

[matthieudelaro]

Here is a ready-to-compile caffe, including the implementation by @daerduoCarey:
https://github.com/matthieudelaro/caffeBVLCplus/tree/stn (stn branch!)
I put files where they belong, modified caffe.proto and filler.hpp, update files to the structural changes of Caffe, un-locked CPU implementation of STN, etc.

[rremani]

@matthieudelaro I want to build the spatial transform layer with py-faster-rcnn. Can you list me the steps.
Thanks

[whuhxb]

@matthieudelaro Have you successfully added the SpatialTransformerLayer with py-faster-rcnn?

[yanxp]

@matthieudelaro @whuhxb Have you successfully added the SpatialTransformerLayer with py-faster-rcnn?

[whuhxb]

Hi:
I'm now trying STN with image recognition, but not yet apply it with py-faster-rcnn. How about you?

At 2017-08-12 16:00:22, "yanxp" [e-mail] wrote:

@matthieudelaro Have you successfully added the SpatialTransformerLayer with py-faster-rcnn?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[mamunir]

Any resource of spatial transformer network in py faster rcnn is appreciated. Thanks

@yanxp @whuhxb @matthieudelaro