One: Compiler FE: RNN and LSTMCell

[seanshpark]

Let's enable RNN and LSTMCell two operators that was forgotten...

• RNN: #1712
• LSTMCell: missed somehow (LSTM at #1477)

For #1928

[seanshpark]

Like #4526, made TF example script

import tensorflow as tf

sess = tf.Session()

in_ = tf.compat.v1.placeholder(dtype=tf.float32, shape=(1, 2, 3), name="Hole")
op_ = tf.keras.layers.RNN(tf.keras.layers.LSTMCell(4))
ex_ = op_(in_)
out = tf.identity(ex_, name="Output")

# we need to save checkpoint to freeze dropped model
init = tf.global_variables_initializer()
sess.run(init)

saver = tf.train.Saver()
saver.save(sess, './ckpt/LSTMCell.ckpt')

# use below command to freeze this model after running tfpem.py
'''
freeze_graph --input_graph LSTMCell.pbtxt \
--input_binary=false \
--input_checkpoint=./ckpt/LSTMCell.ckpt \
--output_node_names=Output \
--output_graph LSTMCell_fr.pbtxt
'''

But drops IndexError: list index out of range error when run freeze_graph

Environment: TF 1.13.x

[seanshpark]

CC @lucenticus , please continue with these Op :)

[seanshpark]

Although not freezed model (with variables) this command can convert to circle file with --v2 option

one-import-tf --input_path LSTMCell.pbtxt --output_path LSTMCell.circle --input_arrays Hole --output_arrays Output --v2

I think there are no additional Ops to enable: as you can see from the circle dump:

Operator Codes: [order] OpCodeName (OpCode Enum)
[0] CUSTOM(VarHandleOp) (code: 32, version: 1)
[1] CUSTOM(TensorArrayV3) (code: 32, version: 1)
[2] TRANSPOSE (code: 39, version: 1)
[3] CUSTOM(TensorArrayScatterV3) (code: 32, version: 1)
[4] WHILE (code: 119, version: 1)
[5] CUSTOM(TensorArraySizeV3) (code: 32, version: 1)
[6] RANGE (code: 96, version: 1)
[7] CUSTOM(TensorArrayGatherV3) (code: 32, version: 1)
[8] STRIDED_SLICE (code: 45, version: 1)
[9] LESS (code: 58, version: 1)
[10] LOGICAL_AND (code: 86, version: 1)
[11] ADD (code: 0, version: 1)
[12] CUSTOM(ReadVariableOp) (code: 32, version: 1)
[13] FULLY_CONNECTED (code: 9, version: 1)
[14] CUSTOM(TensorArrayReadV3) (code: 32, version: 1)
[15] MUL (code: 18, version: 1)
[16] MINIMUM (code: 57, version: 1)
[17] MAXIMUM (code: 55, version: 1)
[18] TANH (code: 28, version: 1)
[19] CUSTOM(TensorArrayWriteV3) (code: 32, version: 1)

~So, solving this freeze step can confirm these two Ops: RNN and LSTMCell :)~

There was one more missing Op :(

• Added #4542 issue

[seanshpark]

But drops IndexError: list index out of range error when run freeze_graph

@lucenticus , to be sure, I'm not working on this issue so if you need this, please go on with this issue.

[lucenticus]

@seanshpark Okay, we'll take a look, thanks for clarifying.

[binarman]

@seanshpark
Thank you for the research

If got it right we have several issues with TF model:

• Can not freeze graph (I tried to hack freeze_graph utility, but it is not that simple)
• CUSTOM(VarHandleOp)
• CUSTOM(TensorArrayV3)
• CUSTOM(TensorArrayScatterV3)
• CUSTOM(TensorArraySizeV3)
• CUSTOM(TensorArrayGatherV3)
• CUSTOM(ReadVariableOp)
• CUSTOM(TensorArrayReadV3)
• CUSTOM(TensorArrayWriteV3)

I suppose all custom operators are related to control flow graph generated from LSTM code

[binarman]

@seanshpark
I have a question about script you provide: is it our target code, or we can use other ways to get LSTM operatos.

I am asking because I tried to generate lstm model in keras (tf 1.13.2):

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

model = keras.Sequential()

model.add(layers.LSTM(2, input_shape=(4,4,)))

model.save("keras_model_lstm.h5")

Then convert it into tflite using script (tf 2.0):

import tensorflow as tf
import tensorflow.keras as keras

model = keras.models.load_model("keras_model_lstm.h5")
model.save("saved_model_lstm")

tflite_model_converter = tf.lite.TFLiteConverter.from_saved_model("saved_model_lstm")
tflite_model_converter.experimental_new_converter = True
tflite = tflite_model_converter.convert()

with open("lstm.tflite","wb") as f:
  f.write(tflite)

This sequence worked without errors and generated tflite model without custom operators.

[seanshpark]

I suppose all custom operators are related to control flow graph generated from LSTM code

Yes... I think these custom operators will be removed when freezed...

[seanshpark]

I have a question about script you provide: is it our target code, or we can use other ways to get LSTM operatos.

it is just for testing... no target model yet... so would say "or we can use other ways to get LSTM operatos." is OK.

This sequence worked without errors and generated tflite model without custom operators.

This is great! Purpose of LSTM validation right now is your objective of this year at the moment (as I know) and how you validate the objective is up to you :)
We can continue work on this for ONE is after we enable keras model import in tf2tfliteV2.py tool.

[binarman]

Just for history:

generate lstm

import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.LSTM(2, input_shape=(4,4)))
model.save("lstm.h5")

generate rnn

import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.SimpleRNN(2, input_shape=(4,4,)))
model.save("rnn.h5")

convert h5 to tflite

import tensorflow as tf
import tensorflow.keras as keras
import sys
import os
import tempfile

if __name__ == "__main__":
  if len(sys.argv) != 2:
    print("need an argument: path to keras model")
    exit()

  model_path = sys.argv[1]

  model = keras.models.load_model(model_path)

  tmpd = tempfile.TemporaryDirectory().name
  model.save(tmpd)

  tflite_model_converter = tf.lite.TFLiteConverter.from_saved_model(tmpd)

  #tflite_model_converter.allow_custom_ops = True
  tflite_model_converter.experimental_new_converter = True

  tflite = tflite_model_converter.convert()

  tflite_model_path = os.path.splitext(os.path.basename(model_path))[0] + ".tflite"
  with open(tflite_model_path,"wb") as f:
    f.write(tflite)

[binarman]

Ok, I checked recurrent layers support more carefully
As I wrote in https://github.com/Samsung/ONE/pull/4698#discussion_r511575891 there were no need to modify tf2tflite in the first place.

Here is the table that shows how TFLite supports RNN operators. (simple models, see previous comment)

legend:
:red_circle: - can not convert model
:yellow_circle: - model converts, but interpreter can not inference it
:green_circle: - model convert, inference successful

Model | Converter | TF 2.0.0beta | TF 2.0.0 | TF 2.0.1 | TF 2.0.3 | TF 2.1.0 | TF 2.1.1 | TF 2.2.0 | TF 2.2.1 | TF 2.3.0 | TF 2.3.1
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
LSTM | v1 | :red_circle: | :red_circle: | :red_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:
\ | v2 | :red_circle: | :red_circle: | :red_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:
RNN | v1 | :red_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:
\ | v2 | :red_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:
GRU | v1 | :red_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:
\ | v2 | :red_circle: | :yellow_circle: | :yellow_circle: | :yellow_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle: | :green_circle:

After I got correct tflite files I tried to convert them to circle and run them using luci interpreter:

1. tflite2circle and circle2circle worked with no problem
2. luci interpreter gave me results equal to original keras models (luci interpreter lack some kernels, so I implemented them in my repo, see https://github.com/binarman/ONE/tree/tf2tflite2/interpreter_validate)

So I think we can consider lstm and rnn layers supported in ONE Compiler via:
keras h5 model (TF 1.13.1) -> tf2tflite --v2 (with at least TF 2.1) -> other ONE tools

@seanshpark
One last thing I want to ask is: should I put keras examples in tf examples or somewhere else?

[seanshpark]

One last thing I want to ask is: should I put keras examples in tf examples or somewhere else?

Please put them in compiler/res/TensorFlowPythonExamples with a comment if there is some special steps required

[binarman]

Nothing to do for this task left, closing
@seanshpark Thank you!