One: [onert] int64 support needed for some op

[hyunsik-yoon]

We need int64 support for the following to run out test model, large_test.tflite:
(large_test.tflite info)

Can someone check and help?

• [x] matrixBandPart
• [x] ArgMax
• [x] Cast

Operator 20: CUSTOM(MatrixBandPart)
        Fused Activation: NONE
        Input Tensors[136, 3, 4]
                Tensor  136 : buffer  137 |  Empty | FLOAT32 | Memory 4.0B   | Shape Scalar (b'transformer_c3po/attention_bias_lower_triangle/attention_bias_local/ones')
                Tensor    3 : buffer    4 | Filled | INT64   | Memory 8.0B   | Shape Scalar (b'transformer_c3po/attention_bias_lower_triangle/attention_bias_local/Cast')
                Tensor    4 : buffer    5 | Filled | INT64   | Memory 8.0B   | Shape Scalar (b'transformer_c3po/attention_bias_lower_triangle/attention_bias_local/Cast_1')
        Output Tensors[137]
                Tensor  137 : buffer  138 |  Empty | FLOAT32 | Memory 4.0B   | Shape Scalar (b'transformer_c3po/attention_bias_lower_triangle/attention_bias_local/MatrixBandPart')

Operator 907: ARG_MAX
        Fused Activation: NONE
        Input Tensors[1063, 48]
                Tensor 1063 : buffer 1957 |  Empty | FLOAT32 | Memory 4.0B   | Shape Scalar (b'transformer_c3po/while/Squeeze')
                Tensor   48 : buffer  942 | Filled | INT32   | Memory 4.0B   | Shape Scalar (b'transformer_c3po/get_timing_signal_1d/range/delta1')
        Output Tensors[1064]
                Tensor 1064 : buffer 1958 |  Empty | INT64   | Memory 8.0B   | Shape Scalar (b'transformer_c3po/while/ArgMax')

Operator 908: CAST
        Fused Activation: NONE
        Input Tensors[1064]
                Tensor 1064 : buffer 1958 |  Empty | INT64   | Memory 8.0B   | Shape Scalar (b'transformer_c3po/while/ArgMax')
        Output Tensors[1065]
                Tensor 1065 : buffer 1959 |  Empty | INT32   | Memory 4.0B   | Shape Scalar (b'transformer_c3po/while/ToInt32')

[ragmani]

2277 introduces INT64 type into onert except nnfw API and nn API because CUSTOM(MatrixBandPart)'s int64 operands are constant and int64 operands of ARG_MAX, CAST are not inputs or outputs of model.

[intom]

2339 add int64 for cast operation.

It seems that nnapi doesn't support int64.
How can I test with *.mod.py?

[hyunsik-yoon]

@intom,
Then we may need to create a test tflite file.
I will create a guide issue for others.

[ragmani]

@intom
FYI:
The nnfw api hasn't supported int64 type. So if you want to test it with tflite files, you should introduce the type into nnfw api or create a test tflite file that int64 type are not inputs or outputs of model.

[hyunsik-yoon]

you should introduce the type into nnfw api
or create a test tflite file that int64 type are not inputs or outputs of model

I guess the first way is easier... for the first way.
as I understand, @ragmani's first way is as follows. (@ragmani could you please double-check?)

• add int64 type into nnfw.api
• tflite with int64 input or ourput
• then write nnfw api test to run the tflite file

[ragmani]

@hyunsik-yoon
Yes. If you want to run the model with nnpackage_run, you should add one more thing the following

• add int64 type into nnpackage_run and nnfw::misc::RandomGenerator

[hyunsik-yoon]

@intom, It's getting bigger. :-) Can you handle the test?
If you think it would be better to be handled by others, please let me know.

[intom]

@intom, It's getting bigger. :-) Can you handle the test?
If you think it would be better to be handled by others, please let me know.

I am trying to make tflite file but I meet the error relating to docker.
If it is time critical, I think that it would be better to be handled by others.

[hyunsik-yoon]

@intom

then write nnfw api test to run the tflite file

• for nnfw api test. please refer to gtest files in tests/nnfw_api/src
• after writing your test, you need to make package zip file into uploaded by @wateret or @hseok-oh
• also let test driver know your nnpackage needs to be downloaded
  
  
  
  • you need to create a config.sh. please refer to tests/scripts/nnfw_api_gtest/models/unknown_dim_input_concat/config.sh
  
  
• when test is written and package zip is written, run
  ./infra/scripts/test_ubuntu_runtime_mixed.sh, which will download the package files and run nnfw api gtest.

[glistening]

@ragmani, @hyunsik-yoon If no one work for nnpackage_run int64, I will do. random input generation and h5 dump/load are required.

[ragmani]

@glistening
The work is assigned to @intom . He is working on it.

@intom
According to the @glistening 's mention, the work requires h5 dump/load too.

[hyunsik-yoon]

related works were done.