One: Compiler FE: Fuse InstanceNorm 002

Created on 3 Jun 2020 · 13Comments · Source: Samsung/ONE

Background

Current TensorFlow produces a small network for InstanceNorm op.

Related issue: #1741

In our circle we introduced an OpCode for this as BuiltinOperator_INSTANCE_NORM

_luci_ provides a Pass to fuse this small network into CircleInstanceNorm IR

FuseInstanceNormPass.cpp

Current Pass supports old(?) network when we started last year

Latest TensorFlow produces a bit different network

link to tflite recipe for this: Net_InstanceNorm_002
with python script https://github.com/Samsung/ONE/issues/1741#issuecomment-637188577

Task

We also need to fuse Net_InstanceNorm_002 into CircleInstanceNorm IR

Implement in FuseInstanceNormPass.cpp file

How to check if works

dredd test for this should pass with this change

diff --git a/compiler/circle2circle-dredd-recipe-test/test.lst b/compiler/circle2circle-dredd-recipe-test/test.lst
index c236a8e..9038119 100644
--- a/compiler/circle2circle-dredd-recipe-test/test.lst
+++ b/compiler/circle2circle-dredd-recipe-test/test.lst
@@ -8,5 +8,5 @@
 # resolve_customop_batchmatmul

 Add(Net_InstanceNorm_001 PASS fuse_instnorm)
-# Add(Net_InstanceNorm_002 PASS fuse_instnorm)
+Add(Net_InstanceNorm_002 PASS fuse_instnorm)
 Add(BatchMatMulV2_000 PASS resolve_customop_batchmatmul)

dredd rule for this: - Net_InstanceNorm_002/test.rule

help wanted

Source

seanshpark

All 13 comments

@seanshpark
Did you start this task?
SRR can help you with this, if you want =)

binarman on 9 Jun 2020

Did you start this task?

Nope :)

SRR can help you with this, if you want =)

This is my wish :)

seanshpark on 9 Jun 2020

👍1

Example for InstanceNorm.

import tensorflow as tf
import tensorflow_addons as tfa
from tensorflow.keras import layers

print(tfa.__version__)
print(tf.__version__)

model = tf.keras.Sequential()

model.add(tf.keras.layers.Input(shape=(32,32,8), name="input_layer"))
model.add(tfa.layers.InstanceNormalization(center=True,scale=True,
                                           beta_initializer="random_uniform",
                                           gamma_initializer="random_uniform"))

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.allow_custom_ops = True
tflite_model = converter.convert()
tflite_file = './instance_norm.tflite'
open(tflite_file, "wb").write(tflite_model)

0.10.0
2.2.0

instance_norm

d-krylov on 11 Jun 2020

tflite2circle instance_norm.tflite InstanceNorm.circle
circledump InstanceNorm.circle

Model version: 0
 # sub graphs: 1

Operator Codes: [order] OpCodeName (OpCode Enum)
[0] RESHAPE (code: 22, version: 1)
[1] MEAN (code: 40, version: 1)
[2] SQUARED_DIFFERENCE (code: 99, version: 1)
[3] ADD (code: 0, version: 1)
[4] RSQRT (code: 76, version: 1)
[5] MUL (code: 18, version: 1)
[6] SUB (code: 41, version: 1)

Buffers: B(index) (length) values, if any
B(0) (0) 
B(1) (0) 
B(2) (20) 0x1 00 00 00 0x20 00 00 00 0x20 00 00 00 0x8 00 00 00 ...
B(3) (16) 0x1 00 00 00 0x20 00 00 00 0x20 00 00 00 0x8 00 00 00 ...
B(4) (32) 0x60 0x4c 0xd4 0x3a 0x67 0x8a 0x44 0x3d 0xaa 0x6e 0x2c 0xbd 0xce 0xcf 0xfd 0x3c ...
B(5) (32) 0xc0 0x7b 0xe 0xbd 0x18 0xc4 0x21 0x3c 0xc0 0x7f 0x15 0xbc 0x80 0xc9 0x52 0x3a ...
B(6) (4) 0x6f 0x12 0x83 0x3a
B(7) (12) 0x1 00 00 00 0x2 00 00 00 0x4 00 00 00 ...
B(8) (0) 
B(9) (0) 
B(10) (0) 
B(11) (0) 
B(12) (0) 
B(13) (0) 
B(14) (0) 
B(15) (0) 
B(16) (0) 
B(17) (0) 
B(18) (0) 
B(19) (0) 
B(20) (16) 0x31 0x2e 0x31 0x33 0x2e 0x31 00 00 00 00 00 00 00 00 00 00 ...

-------------------------------------------------------------------
Sub-Graph: #0 main

Data Format:
CHANNEL_LAST (NHWC for 2d, NDHWC for 3d data)

Operands: T(subgraph index : tensor index) TYPE (shape) (shape_signature) B(buffer index) OperandName
T(0:0) FLOAT32 (1, 32, 32, 8) B(1) input_layer
T(0:1) INT32 (5) B(2) sequential/instance_normalization/stack
T(0:2) INT32 (4) B(3) sequential/instance_normalization/Shape
T(0:3) FLOAT32 (1, 1, 1, 8, 1) B(4) sequential/instance_normalization/Reshape_1
T(0:4) FLOAT32 (1, 1, 1, 8, 1) B(5) sequential/instance_normalization/Reshape_2
T(0:5) FLOAT32 () B(6) sequential/instance_normalization/batchnorm/add/y
T(0:6) INT32 (3) B(7) sequential/instance_normalization/moments/variance/reduction_indices
T(0:7) FLOAT32 (1, 32, 32, 8, 1) B(8) sequential/instance_normalization/Reshape
T(0:8) FLOAT32 (1, 1, 1, 8, 1) B(9) sequential/instance_normalization/moments/mean
T(0:9) FLOAT32 (1, 32, 32, 8, 1) B(10) sequential/instance_normalization/moments/SquaredDifference
T(0:10) FLOAT32 (1, 1, 1, 8, 1) B(11) sequential/instance_normalization/moments/variance
T(0:11) FLOAT32 (1, 1, 1, 8, 1) B(12) sequential/instance_normalization/batchnorm/add
T(0:12) FLOAT32 (1, 1, 1, 8, 1) B(13) sequential/instance_normalization/batchnorm/Rsqrt
T(0:13) FLOAT32 (1, 1, 1, 8, 1) B(14) sequential/instance_normalization/batchnorm/mul
T(0:14) FLOAT32 (1, 32, 32, 8, 1) B(15) sequential/instance_normalization/batchnorm/mul_1
T(0:15) FLOAT32 (1, 1, 1, 8, 1) B(16) sequential/instance_normalization/batchnorm/mul_2
T(0:16) FLOAT32 (1, 1, 1, 8, 1) B(17) sequential/instance_normalization/batchnorm/sub
T(0:17) FLOAT32 (1, 32, 32, 8, 1) B(18) sequential/instance_normalization/batchnorm/add_1
T(0:18) FLOAT32 (1, 32, 32, 8) B(19) Identity

Operators: O(subgraph index : operator index) OpCodeName 
    Option(values) ... <-- depending on OpCode
    I T(tensor index) OperandName <-- as input
    O T(tensor index) OperandName <-- as output
O(0:0) RESHAPE 
    I T(0:0) input_layer
    I T(0:1) sequential/instance_normalization/stack
    O T(0:7) sequential/instance_normalization/Reshape
O(0:1) MEAN 
    I T(0:7) sequential/instance_normalization/Reshape
    I T(0:6) sequential/instance_normalization/moments/variance/reduction_indices
    O T(0:8) sequential/instance_normalization/moments/mean
O(0:2) SQUARED_DIFFERENCE 
    I T(0:7) sequential/instance_normalization/Reshape
    I T(0:8) sequential/instance_normalization/moments/mean
    O T(0:9) sequential/instance_normalization/moments/SquaredDifference
O(0:3) MEAN 
    I T(0:9) sequential/instance_normalization/moments/SquaredDifference
    I T(0:6) sequential/instance_normalization/moments/variance/reduction_indices
    O T(0:10) sequential/instance_normalization/moments/variance
O(0:4) ADD 
    Activation(NONE) 
    I T(0:10) sequential/instance_normalization/moments/variance
    I T(0:5) sequential/instance_normalization/batchnorm/add/y
    O T(0:11) sequential/instance_normalization/batchnorm/add
O(0:5) RSQRT 
    I T(0:11) sequential/instance_normalization/batchnorm/add
    O T(0:12) sequential/instance_normalization/batchnorm/Rsqrt
O(0:6) MUL 
    Activation(NONE) 
    I T(0:12) sequential/instance_normalization/batchnorm/Rsqrt
    I T(0:3) sequential/instance_normalization/Reshape_1
    O T(0:13) sequential/instance_normalization/batchnorm/mul
O(0:7) MUL 
    Activation(NONE) 
    I T(0:7) sequential/instance_normalization/Reshape
    I T(0:13) sequential/instance_normalization/batchnorm/mul
    O T(0:14) sequential/instance_normalization/batchnorm/mul_1
O(0:8) MUL 
    Activation(NONE) 
    I T(0:8) sequential/instance_normalization/moments/mean
    I T(0:13) sequential/instance_normalization/batchnorm/mul
    O T(0:15) sequential/instance_normalization/batchnorm/mul_2
O(0:9) SUB 
    Activation(NONE) 
    I T(0:4) sequential/instance_normalization/Reshape_2
    I T(0:15) sequential/instance_normalization/batchnorm/mul_2
    O T(0:16) sequential/instance_normalization/batchnorm/sub
O(0:10) ADD 
    Activation(NONE) 
    I T(0:14) sequential/instance_normalization/batchnorm/mul_1
    I T(0:16) sequential/instance_normalization/batchnorm/sub
    O T(0:17) sequential/instance_normalization/batchnorm/add_1
O(0:11) RESHAPE 
    I T(0:17) sequential/instance_normalization/batchnorm/add_1
    I T(0:2) sequential/instance_normalization/Shape
    O T(0:18) Identity

Inputs/Outputs: I(input)/O(output) T(tensor index) OperandName
I T(0:0) input_layer
O T(0:18) Identity

d-krylov on 11 Jun 2020

For recipe:

tflchef/tools/reverse/tflchef-reverse instance_norm.tflite test.recipe

d-krylov on 11 Jun 2020

@d-krylov , Is the network same as link to tflite recipe for this: Net_InstanceNorm_001 (the one in the main comment) ?
If not, we need to register another one that you made :)

seanshpark on 11 Jun 2020

It coincides with Net_InstanceNorm_002. I make pass for this. And I write my actions as a reminder for myself.

d-krylov on 12 Jun 2020

👍1

@seanshpark, resolve my doubts, please.
Net_InstanceNorm_001 have tensor shape {1, 8, 6, 12} as input. And fillers for beta and gamma.
Perhaps, Net_InstanceNorm_002 should have shape with rank=4 as input and fillers for beta and gamma?

d-krylov on 15 Jun 2020

Net_InstanceNorm_001 have tensor shape {1, 8, 6, 12} as input.

I think we can revise Net_InstanceNorm_002 to have same input shape as Net_InstanceNorm_001

seanshpark on 15 Jun 2020

@d-krylov , if you had some shape problem with export, please refer #2423.
this fixes new created nodes in Pass not having shape informaton when exported.

seanshpark on 18 Jun 2020

Draft #2449

seanshpark on 24 Jun 2020

Related PRs:

[x] tflite recipe #2872
[x] luci pass #2871

binarman on 9 Jul 2020

👍1

Done. Thank you!!!

seanshpark on 22 Jul 2020

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Compiler FE: support Shape op in luci-interpreter

mhs4670go · 4Comments

[cker/ruy] EXPERIMENTAL_RUY_FEATURE flag not working for android

periannath · 3Comments

Channel-wise INT16 quantization support

jinevening · 3Comments

How can I get ruy to use multiple cores?

ragmani · 4Comments

[onert] Improve performance of WICPlanner

periannath · 3Comments