Incubator-mxnet: How to create a Custom Operator with extra parameters in Python?

[zhengxiawu]

say if I have a layer need some input parameters e.g. size = 10 .
How to set that in Custom Operator? I didnt see any examples in
http://mxnet.io/how_to/new_op.html

[saicoco]

You can set them like defining a python class in __init__.For example:

class Softmax(mx.operator.CustomOp):

    def __init__(self, xxx, yyy):
        self.xxx = xxx
        self.yyy = yyy
    def forward(self, is_train, req, in_data, out_data, aux):
        x = in_data[0].asnumpy()
        y = np.exp(x - x.max(axis=1).reshape((x.shape[0], 1)))
        y /= y.sum(axis=1).reshape((x.shape[0], 1))
        print self.xxx, self.yyy
        self.assign(out_data[0], req[0], mx.nd.array(y))

    def backward(self, req, out_grad, in_data, out_data, in_grad, aux):
        l = in_data[1].asnumpy().ravel().astype(np.int)
        y = out_data[0].asnumpy()
        y[np.arange(l.shape[0]), l] -= 1.0
        self.assign(in_grad[0], req[0], mx.nd.array(y))

@mx.operator.register("softmax")
class SoftmaxProp(mx.operator.CustomOpProp):
    def __init__(self, xxx, yyy):
        super(SoftmaxProp, self).__init__(need_top_grad=False)

        # add parameter
        self.xxx = xxx
        self.yyy = yyy
    def list_arguments(self):
        return ['data', 'label', 'xxx', 'yyy']

    def list_outputs(self):
        return ['output']

    def infer_shape(self, in_shape):
        data_shape = in_shape[0]
        label_shape = (in_shape[0][0],)
        output_shape = in_shape[0]
        return [data_shape, label_shape], [output_shape], []

    def create_operator(self, ctx, shapes, dtypes):
        return Softmax(xxx=self.xxx, yyy=self.yyy)

Here is a example about Softmax, we can see that there are not any parametres in __init__, it just extend its father class, if you wanna add some parameter, you can set them in __init__, and this is same as creating op by extending mx.operator.NDArrayOp. Just see that like a simple python class.

Also, if you use them in forward() or backward(), you can set them to class Softmax like above code.

And when you use the customOP, you need set arguments names in list_arguments(), then you can input them by arguments when you use them.

[zhengxiawu]

@saicoco it works! thanks a lot!

[saicoco]

nice zan!!!

[pengwangucla]

Is it able to add parameters that are not string, but self defined classes ?

I pass the class in while all the parameters are converted to str @saicoco

[HaozhiQi]

@pengwangucla
You may use cPickle to do that. When defining the symbol, passing parameters by
certain_param=cPickle.dump(self_defined_things)
And inside the operator, using cPickle.loads(self_defined_things)

I use this one when passing easydict object to operator.

[saicoco]

Means that you wanna pass a class as arg to your operator?
@pengwangucla

[Rizhiy]

@Oh233 How do I do that in python 3?

[opringle]

Also struggling to implement this in Python 3.