One: [onert] Scalar/Vector tensor and OperationValidator check

[dayo09]

I have been confused about how to handle scalar and vector input tensor.
This issue is to make some points, please tell me if something's wrong.

Scalar(0-dim) Vector(1-dim) input tensors

• some operations have input tensors of 0-dim or 1-dim.
• 0-dim tensor : loaded as a param, usually int or float. Or loaded as a Tensor
• 1-dim tensor : loaded as a vector, usually vector. Or loaded as a Tensor

Pros and Cons of loading them as param / as tensor

(please tell me if I know wrongly)

as tensor

• different data type can be supported

  as param

• runtime's tensor allocation gets less load

• param values can be used while OperationValidator check (otherwise ConstnatnInit runs after OperationValidator so it cannot be checked)

[dayo09]

Operator Validation during runtime

• Compiler::compile() runs OperationValidator and then CreateExecutor- which contains ConstInit or so.
• So actually some of operation-validating is done during runtime
• (suggestion) Why don't we have OP_REQUIRES-like macro all over the cpu kernels? Will be useful.

[hyunsik-yoon]

_param_ is decided at compilation time, meaning param value is in a model file. (e.g., const..)
_tensor_ can be read at compilation (when it is a const) time but also also be read at runtime (output of op).
In other words, in pbtxt file, there are some attributes, e.g., strides is always provided by model file and it is also encoded into tflite file, so that is treated as param. However ker is _input_, which could be a scalar, vector, or 2d tensor. Regardless of its rank, _input_ needs to be treated as _tensor_ rather than _param_.

node {
  name: "ofm"
  op: "Conv2D"
  input: "ifm"
  input: "ker"
  attr {
    key: "T"
    value { type: DT_FLOAT }
  }
  attr {
    key: "data_format"
    value { s: "NHWC" }
  }
  attr {
    key: "dilations"
    value {
      list { i: 1 i: 1 i: 1 i: 1 }
    }
  }
  attr {
    key: "padding"
    value { s: "VALID" }
  }
  attr {
    key: "strides"
    value {
      list { i: 1 i: 1 i: 1 i: 1 }
    }
  }
}

[dayo09]

@hyunsik-yoon Thank you for the explanation between Tensor and Param. However I meant somewhat differnet. I think my explanation was vague.
In this issue, I wanted to talk about IR's param, not about modelfile's tensor and param(attribute).

For example, OneHot operation, loader reads ON_VALUE tensor into fload on_value param.
I mean, loading input into Input::ON_VALUE or Param::on_value is decided on loader implementation.

class OneHot : public Operation
{
public:
  enum Input
  {
    INDICES = 0,
    DEPTH = 1,
    ON_VALUE = 2,
    OFF_VALUE = 3,
  };

  struct Param
  {
    int depth;       // comes from input tensor, not from OneHotOptions
    float on_value;  // comes from input tensor, not from OneHotOptions
    float off_value; // comes from input tensor, not from OneHotOptions
    int axis;
  };

Some implementations puts them into Param and others puts into Input. I want to figure out the guideline here.

[glistening]

@dayo09 I think OneHot implementation assumes on_value and off_value is constant (determined at loading time). In addition, data type of on_value and off_value could be different type like int32. OneHot need to be updated.

[dayo09]

@glistening

This is summary of our off-line conversation, which can somewhat answer some issues here.

Actual Cases

• Cases of loading Tensor into scalar/vector
  
  
  
  • Reduce operation's axis
  
  
  • Transpose operation's perm
  
  
  • Onehot operation's depth, on_value, off_value
  
  
• Cases of loading scalar/vector into Tensor
  
  
  
  • NONE
  
  

Points

The effect of loading Tensor and into scalar/vector is,

• __Cannot handle non-constant input★__
• Cannot handle different variable types

• Reduce operation's axis, Transpose operation's perm
  
  
  
  • constant? constant only cases so far
  
  
  • type? integer only
  
  
• Onehot operation's depth, on_value, off_value :
  
  
  
  • constant? constant only cases so far
  
  
  • type? interger/float -> have to be changed into Tensor to support different types
  
  

[hyunsik-yoon]

You may already know, but just in case. @ragmani is recently working on converting axis param to axis tensor.

[ragmani]

This is summary of our off-line conversation, which can somewhat answer some issues here.

I think we should consider this as three components: frontend, ir, backend

• Frontend : tflite and circle's schema

  • Builtin options: always constant
  • Inputs : constant or non-constant

• IR

  • Params : always constant, This cannot be tensor in our runtime now
  • Inputs : constant or non-constant, This can be tensor in our runtime now

• Backend: kernels

  • Params : always constant
  • Tensors : constant or non-constant

[dayo09]

@ragmani
Thank you for the explanation. Params(builtin options) are constant and inputs(tensors) are either constant or non-constant.
I assume that until now there were hardly cases of axis coming as non-constant, but now you are changing them into tensor to support non-constant axis input.
I wonder if change in transpose's perm is needed also.

[hyunsik-yoon]

I wonder if change in transpose's perm is needed also.

Yes, we may need perm as tensor someday.

[ragmani]

I wonder if change in transpose's perm is needed also.

Of course, I'm sure it would be needed.

[dayo09]

Hereby my curiosity is solved. Closing the issue.

[dayo09]

• [ ] TODO : changing perm in Transpose operatition from param to Tensor