One: Decode custom_options in tflite or circle file

[mhs4670go]

I have Custom OP supported recently. For this, I read Custom op information;custom_options. These are some of those.(https://github.com/Samsung/ONE/issues/11#issue-601650936)

Read CustomOp information

BatchMatMulV2 has four args - x, y, adj_x, adj_y
To get meaningful information from custom ops, I got custom attribute according to Args status.
1. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_) with float32
- Custom attribute : 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 84, 0, 3, 3, 16, 11, 3, 1, 3, 0, 0, 0, 4, 104, 104, 6, 36, 1

1. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_) with int32

2. Custom attribute : 84, 0, 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 3, 15, 14, 9, 3, 1, 3, 2, 0, 0, 4, 104, 104, 6, 36, 1

3. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_,name="bat") with float32

4. Custom attribute : 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 84, 0, 3, 3, 16, 11, 3, 1, 3, 0, 0, 0, 4, 104, 104, 6, 36, 1

5. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_,adj_x=True) with float32

6. Custom attribute : 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 84, 0, 3, 3, 16, 11, 3, 1, 3, 0, 1, 0, 4, 104, 104, 6, 36, 1

7. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_,adj_y=True) with float32

8. Custom attribute : 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 84, 0, 3, 3, 16, 11, 3, 1, 3, 0, 0, 1, 4, 104, 104, 6, 36, 1

9. bmm_ = tf.compat.v1.raw_ops.BatchMatMulV2(x=reshape_,y=bmm_rhs_,adj_y=True, adj_x=True) with float32

10. Custom attribute : 97, 100, 106, 95, 120, 0, 97, 100, 106, 95, 121, 0, 84, 0, 3, 3, 16, 11, 3, 1, 3, 0, 1, 1, 4, 104, 104, 6, 36, 1

So, I can get information of adj_x and adj_y from above payload.

• Attribute[21] : TensorType
• Attribute[22] : adj_x
• Attribute[23] : adj_y

97(a), 100(d), 106(j), 95( _ ), 120(x), 0, 97(a), 100(d), 106(j), 95( _ ), 121(y), 0, 84(T)

I can access its attribute by accessing Attribute[22] or sth and it works well:) But, it is too specific I think. So, I want to find a way to embrace every custom op.

I would like to ask your advice:)

[seanshpark]

How does TensorFlow lite handle CustomOp?

[mhs4670go]

What is flexbuffers

TFLC generates custom_options using flexbuffers.

Flexbuffers is a flex(?) version of Flatbuffers. When you want to store data that doesn't fit a schema, you can use this.

What to convert

The custom_options is a data that stores the attribute of the custom op. For example, the "All" op is supported by TensorFlow, but not TFLite, which is why TFLC converts "All" op into custom ops.

These are its specification.

REGISTER_OP("All")
    .Input("input: bool")
    .Input("reduction_indices: Tidx")
    .Output("output: bool")
    .Attr("keep_dims: bool = false") // note here
    .Attr("Tidx: {int32, int64} = DT_INT32") // and here
    .SetShapeFn(shape_inference::ReductionShape);

Among these pieces of information, the part in Attr is serialized as custom_options.

How to test the generated custom_options

There are three steps to test.

1. Generate custom_options by TFLC
2. Generate custom_options by myself.
3. Compare!

Generate custom_options by TFLC

I wrote a simple python script to generate All.pbtxt

import tensorflow as tf

input_ = tf.compat.v1.placeholder(dtype=tf.bool, shape=(2, 4), name="Hole")
op_ = tf.raw_ops.All(input=input_, axis=1,keep_dims=False)

Then, I converted it to TFLite with tf2tfliteV2.

$ python ~/ONE/build/compiler/tf2tfliteV2/tf2tfliteV2.py \
--v2 --input_path All.pbtxt --output_path All.tflite \
--input_arrays Hole,All/reduction_indices --output_arrays All

NOTE You must use 2.X TFLC with --v2 option. 1.X doesn't generate its custom_options.

• custom_options : 84, 105, 100, 120, 0, 107, 101, 101, 112, 95, 100, 105, 109, 115, 0, 2, 16, 12, 2, 1, 2, 2, 0, 4, 104, 4, 36, 1

Generate custom_options by myself

I wrote a simple c++ script to generate custom_options.

// Create flexbuffers Builder
auto flex_builder = new flexbuffers::Builder();
size_t map_start = flex_builder->StartMap();

flex_builder->Int("Tidx",tflite::TensorType_INT32); // .Attr("Tidx: {int32, int64} = DT_INT32")
flex_builder->Bool("keep_dims", false); // Attr("keep_dims: bool = false")

flex_builder->EndMap(map_start);
flex_builder->Finish();

// Print custom_options
std::cout << "custom_options : ";
for(auto x : flex_builder->GetBuffer())
{
  std::cout << static_cast<int32_t>(x) << ",";
}
std::cout << std::endl;

• custom_options : 84, 105, 100, 120, 0, 107, 101, 101, 112, 95, 100, 105, 109, 115, 0, 2, 16, 12, 2, 1, 2, 2, 0, 4, 104, 4, 36, 1

Compare!

It looks same:)

What needs further investigation

1. Serialization order affects data.

flex_builder->Int("Tidx",tflite::TensorType_INT32);
flex_builder->Bool("keep_dims", false);

And

flex_builder->Bool("keep_dims", false);
flex_builder->Int("Tidx",tflite::TensorType_INT32);

yield different custom_options.

1. How the actual value of attr is serialized

What I am going to do

I am going to make custom_options generator function or sth in tflchef.