Incubator-mxnet: how to use mx.sym.UpSampling for bilinear upsampling

[animebing]

I am trying to use mx.sym.UpSampling for bilinear upsample, but i don't know how to set the parameter. I hope someone can give me a specific example to use it, thanks.

[back2yes]

I ran into an error when using the bilinear upsampling method.
src/operator/./deconvolutional-inl.h:75: Check failed: (pad_y) >= (target_shape[0]) too big target shape

This happened long time ago in #2823 . Recently I found MXNet made some updates about the bilinear upsampling, I thought they might have fixed it. I built the bleeding edge version from source just now, and found the issue was still there.

[animebing]

@back2yes I ran into the same error too, I don't figure out why, but i use deconvolution to implement the upsample operation, i refer to https://github.com/dmlc/mxnet/issues/1514 to use deconvolution. if you use deconvolution, you need to initialize the bilinear weight, otherwise it will be all zero.

[back2yes]

@animebing

Sorry I did not get a notice about your message.

Indeed, I used Deconvolution in the end too. But I just initialized the weights randomly.
Besides, Deconvolution works over all channels to get a single feature map, while bilinear upsampling transforms in a one channel to one channel fashion. So I think my deconvolutional layers worked in a different way from the real bilinear upsampling method.

Did you have a way to configure the weights so that they could work in the same way?

[animebing]

@back2yes I don't make it very clear last time, if you want to use Deconvolution as bilinear upsample, the num_filter and num_group should be same as the number of channel of last layer, then you can do as below

import mxnet as mx
import numpy as np
data = mx.sym.Variable('data') 
f = 2  # f is the upsample factor
upsample = mx.sym.Deconvolution(data=data, kernel=(2*f, 2*f), stride=(f, f), pad=(f//2, f//2), num_filter=3, num_group=3, name="upsample")  # here the shape of data is (1, 3, 2, 2)
print upsample.list_arguments() # there is 'data', 'upsample_weight', no 'upsample_bias'
exe = upsample.simple_bind(ctx=mx.gpu(0), data=(1, 3, 2, 2))  
exe.arg_dict['data'][:] = mx.nd.array(np.random.randn(1, 3, 2, 2))
init = mx.init.Initializer()
init._init_bilinear('upsample_weight', exe.arg_dict['upsample_weight'])
exe.forward()
print exe.outputs[0].asnumpy() # the shape is (1, 3, 4, 4)

above is my code to test whether it works, maybe it can help you

[happygds]

Do you find out how to use the bilinear way in mx.symbol.upsampling() now? For now, I use GridGenerator() and BilinearSampler() instead to bilinearly upsample x .

[lordofgod]

@animebing @piiswrong I find that if you create an bilineaer upsampling operator, it will return a deconvolution operation in srcoperator\upsampling.cc. It's sure that deconvolution can be used as bilinear if we forward a network, only that the upsample_weight is initialized as bilinear kernel. But during backpropogation, the upsampling_weight will be updated. Should I carefully ignore the weight while training a net?

[animebing]

@lordofgod , I am sorry that I can't help you, because I haven't used MXNET for almost one year.

[zhanghang1989]

@szha This issue can be closed in favor of https://github.com/apache/incubator-mxnet/pull/9688

import mxnet as mx
x1 = mx.nd.ones(shape=(2,3,4,4))
y1 = mx.nd.contrib.BilinearResize2D(x1, out_height=5, out_width=5)