Tvm: [RUNTIME] Shape query API in TVM Graph Runtime

[feng98]

The ARM project need shape information of input data, in order to realize flexible parameter configuration. Is there any other convenient way to get the shape information of input data when use runtime in ARM?
@tqchen @merrymercy

[srkreddy1238]

I feel we should have an graph runtime API just to query input and output dimensions.

@tqchen what do you think ?

[tqchen]

Sorry for delayed reply here, yes we want to add runtime interface to query the shape. The real question is what should it look like.

• We could directly create an API that returns a tensor(ndarray) reference
  
  
  
  • Need to change the runtime
  
  
  • Do not work across the RPC boundary
  
  
• Create an information query function that returns dimension and each value of the shape
  
  
  
  • Need additional frontend wrapping
  
  
  • Works across RPC boundary
  
  

@dmlc/tvm-team what do you think

[srkreddy1238]

I vote for second option(frontend wrapping) to keep it common and available every where.

[kevinthesun]

Vote for 2 for RPC support.

[kazum]

+1 for the second one, for the same reason with @srkreddy1238 and @kevinthesun.

[tmoreau89]

Same, option 2 - it's best to keep the RPC stable.

[zhreshold]

+1 for option 2.

[tqchen]

OK, seems we are reaching the consensus of option2. I think we can create a packed function API. I wrote the following thing in python, but we can build a packed function version of this.

def get_input_info(self, name):
    def tensor_info(int field, int index):
        # field is an enum of ndim, shape, dtype
        # index corresponds to the specific shape
    return tensor_info 

[tqchen]

This issue is now marked as help wanted, as we have a candidate solution. if any people would like to propose the change to implement it, please reply and update with C++ code snippet. Thanks for everyone involving the discussion.

[srkreddy1238]

What would be the final return for get_input_info ?
How packed function args & return values ? receives: (name, field, index ) return : (integer) ?

Hope there would be similar for "get_output_info" too ..

[tqchen]

+1 for get_output_info. The get_input_info returns a PackedFunc, which takes in field, index and return the corresponding field.

[srkreddy1238]

@tqchen
I am working on C++ ref code and few queries as listed below.

• Is get_input_into, get_output_info itself is a Packed function which is returned by calling GetFunction (like load_params, get_output ...etc.) ? or We extend Module class to add GetInputInfo, GetOutputInfo ?

• output nodes may have multiple outputs, hence do we add node_index attribute to the packed function def tensor_info(int field, int index): ?

[srkreddy1238]

@tqchen

[tqchen]

I think a bit more about it, maybe a better way is to enable RPC to return a remote NDArray in its return values, it needs a bit to refactor of RPC but doable, I will try a stab on this

[tqchen]

https://github.com/dmlc/tvm/pull/1610 support return NDArray through RPC, so we can directly do get_input_array and get_output_array

[srkreddy1238]

@tqchen
Do we leave existing get_output as is is and go with new API as get_input_array(index) and get_output_array(index) to return NDArray?

[tqchen]

yes, let us do that, however, let us make sure to redirect the calls if can be implemented on top of another

[srkreddy1238]

1659 for review.

[tqchen]

close by #1659