Coremltools: Implemented constant_pad_nd but failed

[mushipand]

❓Question

Hello!
While converting PyTorch model, I implemented 'constant_pad_nd' but I hit Input 'val' of op 'outputs1.1' (const) must be const at compile time.
What is happening?

@register_torch_op(torch_alias=["constant_pad_nd"])
def pad(context, node):
    inputs = ops._get_inputs(context, node, expected=3)
    x = inputs[0]
    lis = inputs[1].val
    lis2 = inputs[2].val
    print(lis, lis2)

    if lis2 == 0:
        if (lis[0] == 0) & (lis[2] == 0):
            x = mb.const(mode='immediate_value', val=x, name=node.name)
        else:
            if lis[0] != 0:
                h = mb.const(mode='immediate_value', val=np.zeros(x.shape[0]*x.shape[1]*x.shape[2]).reshape(x.shape[0],x.shape[1],x.shape[2],lis[0]))
                x = mb.concat(values=[h,x,h], axis=3)
            if lis[2] != 0:
                v = mb.const(mode='file_value', val=np.zeros(x.shape[0]*x.shape[1]*x.shape[3]).reshape(x.shape[0],x.shape[1],lis[2],x.shape[3]))
                x = mb.concat(values=[v,x,v], axis=2)

    context.add(x)

Actually, I do not fully understand immediate_value and file_value, so could you please give me more information about that?

This time, I wanted to make the counterpart for F.pad(input_tensor, [0,0,0,0]).
The node gets three inputs ; tensor, [0,0,0,0], 0. In my understanding, the first tensor represents input_tensor, the second [0,0,0,0], the third value=0, though the third one is omitted in PyTorch implementation.

Thanks.

System Information

torch == 1.6.0
coremltools == 4.0b3

[vidursatija]

Hi,
I don't know what the problem is but I implemented constant_pad_nd as follows:

@register_torch_op
def constant_pad_nd(context, node):
    inputs = _get_inputs(context, node, expected=3)
    # print(inputs[1].val, inputs[0].shape)
    new_pad = inputs[1].val.reshape((-1, 2))[::-1].reshape(-1).tolist()
    new_pad = [0]*(2*len(inputs[0].shape)-len(new_pad)) + new_pad
    print(new_pad, inputs[0].shape)
    padded = mb.pad(x=inputs[0], pad=_np.array(new_pad), mode="constant", constant_val=float(0), name=node.name)
    context.add(padded)

This has worked for me for multi rank tensors with constant 0 padding.

[mushipand]

Hi,
It worked for my situation too! Thank you so much!

[aseemw]

@mushipand , you got that error because in this line the val attribute does not point to a constant value. It should always hold value that is constant at the compile time.

mb.const(mode='immediate_value', val=x, name=node.name)

As for the immediate value vs file value, its a contract in the MIL format, how the constant values are stored. Both of them should work and they are not the source of the error you observed.

[mushipand]

@aseemw ,thank you for your support!
Maybe I misunderstood the meaning 'constant'. Input values changes all the time, so that is never constant. Am I right?

[aseemw]

mb.const is used to insert a constant tensor in the graph, like you are correctly using it here:

h = mb.const(mode='immediate_value', val=np.zeros(x.shape[0]*x.shape[1]*x.shape[2]).reshape(x.shape[0],x.shape[1],x.shape[2],lis[0]))
x = mb.concat(values=[h,x,h], axis=3)

[mushipand]

I see. Thank you so much!