Coremltools: Tensorflow2: Convert LSTM fails

[haifengkao]

🐞Describe the bug

• unable to convert lstm layer

• which converter: tensorflow

Trace

If applicable, please paste the error trace.
Running TensorFlow Graph Passes: 100%|██████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 10.10 passes/s]
Converting Frontend ==> MIL Ops:   0%|                                                                                                | 0/46 [00:00<?, ? ops/sWARNING:root:Input ls elem type unknown. Override with <class 'coremltools.converters.mil.mil.types.type_tensor.tensor.<locals>.tensor'> 0/36 [00:00<?, ? ops/s]
Converting Frontend ==> MIL Ops: 100%|█████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:00<00:00, 4198.27 ops/s]
Converting Frontend ==> MIL Ops: 100%|████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 26366.51 ops/s]
Converting Frontend ==> MIL Ops: 100%|██████████████████████████████████████████████████████████████████████████████████████| 46/46 [00:00<00:00, 948.35 ops/s]
Traceback (most recent call last):
  File "coreml_test.py", line 16, in <module>
    source="tensorflow",
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/_converters_entry.py", line 258, in convert
    **kwargs
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 120, in _convert
    prog = frontend_converter(model, **kwargs)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 52, in __call__
    return tf2_loader.load()
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 79, in load
    program = self._program_from_tf_ssa()
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow2/load.py", line 181, in _program_from_tf_ssa
    return converter.convert()
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 393, in convert
    self.tensorflow_passes(prog)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow2/ssa_passes/tf_passes.py", line 26, in tensorflow_passes
    PASS_REGISTRY[p](prog)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/ssa_passes/backfill_make_list_elem_type.py", line 29, in backfill_make_list_elem_type
    backfill_make_list_elem_type_block(f)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/ssa_passes/backfill_make_list_elem_type.py", line 63, in backfill_make_list_elem_type_block
    name=op.name,
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
    return cls._add_op(op_cls, **kwargs)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/mil/builder.py", line 186, in _add_op
    op_cls.input_spec, kwargs["name"], op_cls, before_op, kwargs
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/mil/builder.py", line 145, in _create_input_vars
    var = cls._add_const(val, new_var_name, before_op)
  File "/opt/anaconda3/lib/python3.7/site-packages/coremltools/converters/mil/mil/builder.py", line 82, in _add_const
    raise ValueError("Cannot add const {}".format(val))
ValueError: Cannot add const (is3, 1280)

To Reproduce

• If a python script can reproduce the error, please paste the code snippet

import coremltools
import tensorflow.keras as keras

input_shape = (1, 1280)
inp = keras.layers.Input(shape=input_shape)
h0 = keras.layers.Input(shape=(512,))
c0 = keras.layers.Input(shape=(512,))
out, hn, cn = keras.layers.LSTM(512,
                                return_sequences=True,
                                return_state=True,
                                recurrent_activation='sigmoid')(inp)
keras_model = keras.models.Model(inputs=[inp, h0, c0], outputs=[out, hn, cn])

model = coremltools.convert(
    keras_model)

System environment (please complete the following information):

• coremltools version (e.g., 3.0b5): 4.0b1
• OS (e.g., MacOS, Linux): MacOS
• macOS version (if applicable): OSX 10.5.5
• XCode version (if applicable): 11.6b
• How you install python (anaconda, virtualenv, system): anaconda
• python version (e.g. 3.7): 3.7
• any other relevant information: tenserflow 2.2

[VictorBulba]

I have this bug too. Maybe someone knows what to do?

[Mstronach]

Hi, I have created a ticket for this so that we will be able to track and resolve this issue in the coming releases. Please feel free to add any other details related to this issue in the mean time.

[jakesabathia2]

@haifengkao I believe this issue has been fix in coremltools 4.0b4, please give it a try! Thanks!

[jakesabathia2]

Please reopen the issue if you still get any problem converting this model :)