Coremltools: error converting a tf.keras RNN model

[DanWBR]

🐞Describe the bug

• model is saved and loaded correctly using tf.keras
• during conversion to coreml model, an exception occurs: RuntimeError: Failed to load SavedModel or .h5 model. Cannot find the Placeholder op that is an input to the ReadVariableOp..

Trace

File "/Library/Python/3.7/site-packages/coremltools/converters/tensorflow/_tf_converter.py", line 119, in _graph_def_from_saved_model_or_keras_model
frozen_func = _convert_to_constants.convert_variables_to_constants_v2(concrete_func)
File "/Library/Python/3.7/site-packages/tensorflow_core/python/framework/convert_to_constants.py", line 650, in convert_variables_to_constants_v2
func, lower_control_flow)
File "/Library/Python/3.7/site-packages/tensorflow_core/python/framework/convert_to_constants.py", line 511, in _convert_variables_to_constants_v2_impl
raise ValueError("Cannot find the Placeholder op that is an input "
ValueError: Cannot find the Placeholder op that is an input to the ReadVariableOp.

System environment (please complete the following information):

• coremltools version (e.g., 3.0b5): 3.1
• OS (e.g., MacOS, Linux): macOS
• macOS version (if applicable): 10.15.1
• How you install python (anaconda, virtualenv, system): system
• python version (e.g. 3.7): 3.7.3
  
  
  
  • tensorflow version: 2.1.0-dev20191203
  
  

model

import os
import time

import numpy as np
import tensorflow as tf
import unidecode
from keras_preprocessing.text import Tokenizer

print(tf.__version__)

file_path = os.path.join(os.path.dirname(__file__), '09-conversational-phrases.txt')

text = unidecode.unidecode(open(file_path).read())

tokenizer = Tokenizer()
tokenizer.fit_on_texts([text])

encoded = tokenizer.texts_to_sequences([text])[0]

vocab_size = len(tokenizer.word_index) + 1

word2idx = tokenizer.word_index
idx2word = tokenizer.index_word

sequences = list()

for i in range(1, len(encoded)):
    sequence = encoded[i - 1:i + 1]
    sequences.append(sequence)

sequences = np.array(sequences)

X, Y = sequences[:, 0], sequences[:, 1]
X = np.expand_dims(X, 1)
Y = np.expand_dims(Y, 1)

# Batch size
BATCH_SIZE = 1

# Buffer size to shuffle the dataset
# (TF data is designed to work with possibly infinite sequences,
# so it doesn't attempt to shuffle the entire sequence in memory. Instead,
# it maintains a buffer in which it shuffles elements).
BUFFER_SIZE = 10000

dataset = tf.data.Dataset.from_tensor_slices((X, Y)).shuffle(BUFFER_SIZE)
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)

def build_model(vocab_size, embedding_dim, rnn_units, batch_size):
  model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim,
                              batch_input_shape=[batch_size, None]),
    tf.keras.layers.LSTM(rnn_units,
                        return_sequences=True,
                        stateful=False,
                        recurrent_activation='sigmoid',
                        recurrent_initializer='glorot_uniform'),
    tf.keras.layers.Dense(vocab_size)
  ])
  return model

embedding_dim = 100

units = 256

model = build_model(vocab_size, embedding_dim, units, BATCH_SIZE)

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

# Directory where the checkpoints will be saved
checkpoint_dir = os.path.join(os.path.dirname(__file__), 'training_checkpoints2')

# Name of the checkpoint files
checkpoint_prefix = os.path.join(checkpoint_dir, "ckpt_{epoch}")

checkpoint_callback=tf.keras.callbacks.ModelCheckpoint(
    filepath=checkpoint_prefix,
    save_weights_only=True)

#model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))

model.summary()

EPOCHS=1

#history = model.fit(dataset, epochs=EPOCHS, callbacks=[checkpoint_callback])

def loss_function(labels, logits):
    return tf.keras.losses.sparse_categorical_crossentropy(labels, logits, from_logits=True)

for epoch in range(EPOCHS):
    start = time.time()

    model.reset_states()

    for (batch, (input, target)) in enumerate(dataset):
        with tf.GradientTape() as tape:

            predictions = model(input)

            target = tf.reshape(target, (-1,))
            loss = loss_function(target, predictions)

            grads = tape.gradient(loss, model.variables)
            model.optimizer.apply_gradients(zip(grads, model.variables))

            if batch % 100 == 0:
                print('Epoch {} Batch {} Loss {:.4f}'.format(epoch + 1, batch, loss.numpy().mean()))

    model.save_weights(os.path.join(checkpoint_dir, "ckpt_" + str(epoch)))

start_string = "i"

input_eval = [word2idx[start_string]]
input_eval = tf.expand_dims(input_eval, 0)

text_generated = ''

hidden = [tf.zeros((1, units))]

predictions = model(input_eval, hidden)

predictions = tf.reshape(predictions, (-1, predictions.shape[2]))

predicted_id = tf.argmax(predictions[-1]).numpy()

text_generated += " " + idx2word[predicted_id]

print(start_string + text_generated)

model.save('model2', save_format='tf')

conversion code

import coremltools
import tensorflow as tf
import os
import tfcoreml

print(tf.__version__)

basedir =  os.path.dirname(__file__)
modelfile = os.path.join(basedir, 'model2')

keras_model = tf.keras.models.load_model(modelfile)

# print input name, output name, input shape
print(keras_model.input.name)
print(keras_model.input_shape)
print(keras_model.output.name)


model = tfcoreml.convert(modelfile,
                         input_name_shape_dict={'embedding_input': (1, None)},
                         output_feature_names=['Identity'],
         minimum_ios_deployment_target='13')

outmodelfile = os.path.join(basedir, 'model2.mlmodel')

model.save(outmodelfile)

[jakesabathia2]

@DanWBR Thanks for providing this great script and we did reproduce your error.
Unfortunately, it is a known issue in Tensorflow that its convert_variables_to_constants_v2 (which we are now using before converting model into coreml format) doesn't support LSTM layers.
However, coremltools does support RNN for tf.1.x :).
Let us know if you have other concerns.

[jakesabathia2]

https://github.com/tensorflow/tensorflow/issues/29674
@DanWBR
This tensorflow issue is tracking the same error.

[DanWBR]

@jakesabathia2 thank you for your help. I'm not working on this model anymore but I'll keep an eye on this issue as I may come back to it in the future.

[jakesabathia2]

https://github.com/tensorflow/tensorflow/issues/36391