One: Compiler FE: Adding U8 recipies

[seanshpark]

As you can see in https://github.com/Samsung/ONE/tree/master/res/TensorFlowLiteRecipes folder, there are sub folders of recipe that we support(some are work in progress).

• recipe is a protocol buffer file, a soure file to generate TF lite file
• tflchef is a tool that actually generates the file
• we use human readable text source so that editing is easy only with a text editor

There are two data types that we support in major: Float32 and Uint8.
All the Ops have Float32. But some have Uint8 recipe.

This task is to create Uint8 recipes that are missing

• https://www.tensorflow.org/lite/performance/quantization_spec shows list of Ops that TensorFlow defined.
• This document provides specifications for Int8 but we use Uint8.
• Related paper with Uint8: https://arxiv.org/abs/1712.05877
  
  
  
  • _Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference_
  
  

Things to do

• Write test.recipe file. You can copy-paste from existing ones
• Add item to https://github.com/Samsung/ONE/blob/master/compiler/luci/tests/test.lst file
  
  
  
  • configure - build - test should pass
  
  
• Fix problems if anything goes wrong
• One recipe at a time, that is one recipe per PR

---

You can also fire an issue (optional), Add for example

[res] Add U8 recipe: ADD

Let's enable U8 recipe for Add Op

This will let others, including me, know that one is already working on it.

Steps (not mandatory but usually this may happen)

• Create an issue with title like [res] Add U8 recipe: ADD for ADD Op, and some descriptions
• It would be nice if you work on one Op at a time to concentrate on the operator
• I'll update the table of this issue that others know you are working on like #1606 (I'll be notified by github from the new issue)
• create a PR and work on this topic; update PR with Work In Progress codes and this may help getting faster feedbacks
• get a review from guys working on this topic, like @jinevening
• apply feedback changes if any
• land the PR and done !

How to test

./nncc configure
./nncc build
./nncc test

[seanshpark]

Operators in https://www.tensorflow.org/lite/performance/quantization_spec

• please add a comment if this table is incorrect and needs update

Operator | status
-- | --
ADD | (exist)
AVERAGE_POOL_2D | @underflow101 (#3429) done
CONCATENATION | (exist)
CONV_2D | (exist)
DEPTHWISE_CONV_2D | (exist)
FULLY_CONNECTED | (exist)
L2_NORMALIZATION | @underflow101 (#3444) done
LOGISTIC | @underflow101 (#3461) done
MAX_POOL_2D | (exist)
MUL | (exist)
RESHAPE | (exist)
RESIZE_BILINEAR | @underflow101 done
SOFTMAX | (exist)
SPACE_TO_DEPTH | @underflow101 (#3583)
TANH | @underflow101 (#3592)
PAD | (exist)
GATHER |
BATCH_TO_SPACE_ND |
SPACE_TO_BATCH_ND |
TRANSPOSE | done
MEAN | (exist)
SUB | (exist)
SUM |
SQUEEZE |
LOG_SOFTMAX | @underflow101 done
MAXIMUM | done
ARG_MAX | (exist)
MINIMUM | done
LESS | done
PADV2 | (float32 not ready yet)
GREATER | done
GREATER_EQUAL | done
LESS_EQUAL | done
SLICE |
EQUAL | done
NOT_EQUAL | done
SHAPE |
QUANTIZE | (float32 not ready yet)

[vasantteja]

Would like to work on this if this is open.

[seanshpark]

@vasantteja Thank you for your interest! Sure, please pick an Op in the list and fire an issue:)
I think I need to prepare a template for this :) For the mean time, this can be used;
for an example;

[res] Add U8 recipe: ADD

Let's enable U8 recipe for Add Op

We can discuss within this issue if you have any questions.

[vasantteja]

@seanshpark Sure. I will start working on this. Can I please connect with you if I have any doubts. Thanks in advance.

[seanshpark]

Can I please connect with you if I have any doubts.

Sure :) feel free to ask any questions related.

[underflow101]

@seanshpark, I would like to solve this issue.
Is there any place I could join in this problem solving?

[seanshpark]

Is there any place I could join in this problem solving?

@underflow101 , thank your for your interest! I think you've already joined :)

I'm not sure what place you mentioned could be... but I think the steps would be like

• Create an issue with title like [res] Add U8 recipe: ADD for ADD Op, and some descriptions
• It would be nice if you work ~one~ on one Op at a time to concentrate on the operator
• I'll update the table of this issue that others know you are working on like #1606 (I'll be notified by github from the new issue)
• create a PR and work on this topic; update PR with Work In Progress codes and this may help getting faster feedbacks
• get a review from guys working on this topic, like @jinevening
• apply feedback changes if any
• land the PR and done !

[underflow101]

@seanshpark , then is it okay for me to work on QUANTIZE op recipe?
If that is okay for you, I will write an issue as you have guided above and start working on it.

[seanshpark]

then is it okay for me to work on QUANTIZE op recipe?
If that is okay for you, I will write an issue as you have guided above and start working on it.

Sure :) please go ahead

[underflow101]

@seanshpark , Thank you for your kind reply :)

[underflow101]

@seanshpark , I would like to solve AveragePooling2D issue as you have advised me on #3426

Thank you.

[underflow101]

@seanshpark , I have finished my work on AveragePool2D_U8_000 (#3439), waiting for merge.
I would like to solve L2_NORMALIZATION for my next task, if it is okay.

Thank you.

[underflow101]

@seanshpark , I would like to solve LOGISTIC for my next task, if it is okay.

Thank you.

[underflow101]

@seanshpark , I would like to solve RESIZE_BILINEAR while Logistic U8 compiler fix is ongoing, if it is okay.

Thank you.

[underflow101]

@seanshpark , I have a question:
Things like SUM operation has Sum_000 & Sum_001.
Does this mean I have to make U8 recipe for each one of them?
Or do I have to make U8 recipe for only Sum_000?

[seanshpark]

Does this mean I have to make U8 recipe for each one of them?
Or do I have to make U8 recipe for only Sum_000?

These multiple recipes are to check diffent parameters.
For U8, I think only 000 is OK.

[underflow101]

These multiple recipes are to check diffent parameters.
For U8, I think only 000 is OK.

Got it! :+1:
Thank you @seanshpark !

[underflow101]

@seanshpark ,
I'm having problems with Resize_Bilinear.

Related Issue: #3497 ,
circle2circle: /home/bear/data/opensource/ONE/compiler/luci/service/src/CircleTypeInferenceRule.cpp:521: virtual loco::DataType {anonymous}::TypeInferenceAlgorithm::visit(const luci::CircleOutput*): Assertion output_dtype == loco::dtype_get(node->from())' failed.

I have put 4 days of effort in this, (manipulating parameters, etc.) but it doesn't work well.
I think it would be better for me to put my effort in another recipe first, and then come back to this.

I will work on generating U8 recipe for SPACE_TO_DEPTH.

Thank you.

[underflow101]

RESIZE_BILINEAR & LOGSOFTMAX done.
Please update table if you have spare time.

Thank you!