One: [circle-quantizer] Support quantization for more operators

[jinevening]

Let's support more operators in circle-quantizer

Q8: q8 quantization logic
Q16: q16 quantization logic
Verifier: Verifies the type/granularity of the quantized Op

First milestone (4/6)

Operator | Q8 | Q16 | Verifier
-- | -- | -- | --
sigmoid | o | o |  o
softmax | o | o |  o
tanh | o | o | o 
permute (transpose) | o | o | o
reshape | o | o | o
slice | o | o | o 
argmax | o | o | o 

Second milestone (4/12)

Operator | Q8 | Q16 | Verifier | Assignee
-- | -- | -- | -- | --
floor | o | o | o | @dayo09  
div | o | o | o | @dayo09  
floor_div | o | o | o | @dayo09  
greater | o  | o  | o | @jinevening
greater_equal | o  | o  | o | @jinevening
logical_or | o  | o | o | @jinevening
not_equal | o | o | o | @jinevening
concat | o | o | o | @jinevening
pad | o | o | o | @jinevening

Third milestone (4/16)

Operator | Q8 | Q16 | Verifier| Assignee
-- | -- | -- | -- | --
elu | o | o | o | @dayo09  
rsqrt | o | o | o | @dayo09
sqrt | o | o | o | @dayo09
pow | o | o | o | @dayo09  
resize_bilinear | o | o | o | @dayo09  
resize_nearestneighbor | o | o |  | @dayo09  
yuv_rgb_scale (FC+Reshape) | - | - | - | -
instance_normalize | o | o | o | @jinevening
local_response_normalize |   |   |  | need discussion
batch_to_space | o  | o  | o | @jinevening
depth_to_space |  o | o  | o | @jinevening
space_to_batch | o  | o  | o | @jinevening
space_to_depth | o  | o  | o | @jinevening
split | o | o | o | @jinevening
strided_slice | o  | o  | o | @jinevening

CC @lemmaa

[dayo09]

@jinevening I will start with not_equal 😄

[dayo09]

@jinevening It seems compiler/luci/pass/src/test/TestIOGraph.h is staled and moved to compiler/luci/testhelper. https://github.com/Samsung/ONE/pull/6335

May I replace it?

[jinevening]

May I replace it?

Of course :)

[seanshpark]

May I replace it?

If you like please go ahead :)
It would be nice to add an issue for this task.

[dayo09]

By the offline conversation with @jinevening

Note

• Quantization Type (lwq, cwq) doesn't mean all the nodes in the graph are the attributes.
• If a graph's quantization type is lwq
  
  
  
  • lwq: most of the node's quantization type are lwq
  
  
  • except for
    
    
    
    
    
    • bool type node (ex: output of comparison)
    
    
    
    • scalar node (ex: stride of StridedSlice)
    
    
    
    • certain type of parameter node
    
    
    
  
  
• If a graph's quantization type is cwq
  
  
  
  • cwq: only the nodes supports cwq quantization type are cwq
  
  
  • (ex: Weight/Bias of Conv/DConv/TConv/FC, Parameter of PReLU, Parameter of InstanceNorm)
  
  
  • lwq: most of the node's quantization type are lwq
  
  
  • except for
    
    
    
    
    
    • bool type node (ex: output of comparison)
    
    
    
    • scalar node (ex: stride of StridedSlice)
    
    
    
    • certain type of parameter node
    
    
    
  
  

Test Spec

(1) Positive Test

• TEST_WITH_GRAPH
  
  
  
  • Test if quantization with the given op, type and granularity runs without error.
  
  

| Type | Granularity |
| ---- | ----------- |
| U8 | LayerWise |
| U8 | ChannelWise |
| S16 | ChannelWise |

(2) Negative Test

• TEST_WITH_WRONG_TYPE
  
  
  
  • Quantize F32 model to Type by QuantizeWithMinMaxPass
  
  
  • Set the quantized model's output as WrongType
  
  
  • Expect the Verifier throws exception
  
  

Type | Granularity | WrongType
-- | -- | --
U8 | LayerWise | S16
U8 | ChannelWise | S16
S16 | ChannelWise | S16

• TEST_WITH_WRONG_GRANULARITY
  
  
  
  • Quantize F32 model to Type by QuantizeWithMinMaxPass
  
  
  • Set the quantized model's output's quantization param invalid - specifically, by adding one more zerop/scale parameter
  
  
  • Expect the Verifier throws exception
  
  

| Type | Granularity |
| ---- | ----------- |
| U8 | LayerWise |
| U8 | ChannelWise |
| S16 | ChannelWise |

[jinevening]

record-minmax currently does not support bool-type input.

Real world model's input is hardly bool type. But in the viewpoint of CI, we will need bool-type input for testing.

I made a draft to support bool type input in record-minmax #6526.

[jinevening]

All done. Thanks @dayo09 :)