Models: Cannot open object detection model using tf.keras.models.load

[ChowderII]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [X] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• [X] I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [X] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz

2. Describe the bug

tf.keras.models.load fails to open the model with error:

Traceback (most recent call last):
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-a1fec4d829cb>", line 1, in <module>
    loaded_model = tf.keras.models.load_model('resources/ssd_mobilenetv2/saved_model')
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\saving\save.py", line 150, in load_model
    return saved_model_load.load(filepath, compile)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\load.py", line 89, in load
    model = tf_load.load_internal(path, loader_cls=KerasObjectLoader)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 552, in load_internal
    export_dir)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\load.py", line 118, in __init__
    super(KerasObjectLoader, self).__init__(*args, **kwargs)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 121, in __init__
    self._load_all()
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 239, in _load_all
    node, setter = self._recreate(proto)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 322, in _recreate
    return factory[kind]()
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 309, in <lambda>
    "user_object": lambda: self._recreate_user_object(proto.user_object),
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 328, in _recreate_user_object
    return self._recreate_base_user_object(proto)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\load.py", line 216, in _recreate_base_user_object
    return revived_cls._init_from_metadata(metadata)  # pylint: disable=protected-access
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\load.py", line 297, in _init_from_metadata
    revived_obj = cls(**init_args)
  File "C:\Users\brackman\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow_core\python\keras\engine\input_layer.py", line 86, in __init__
    batch_size = batch_input_shape[0]
KeyError: 0

3. Steps to reproduce

1. Download the model from the above link.
2. Run this:

import tensorflow as tf
loaded_model = tf.keras.models.load_model('resources/ssd_mobilenetv2/saved_model')
# keep in mind my path is probably different than yours

4. Expected behavior

I expect to have a Keras instance of the model.

5. Additional context

I am quite new to this and I don't quite know what can be useful to you. Let me know if there are additional information I can provide to help you.

I am fairly certain I am not using that correctly. If it helps you understand why I am trying to do this, I am trying to generate an .hdf5 model for another purpose.

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• Mobile device name if the issue happens on a mobile device: N/A
• TensorFlow installed from (source or binary): binary (conda)
• TensorFlow version (use command below): 2.2.0
• Python version: 3.7.7
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA version: release 10.1 V10.1.243,
• cuDNN version: 7.6.5
• GPU model and memory: GeForce GTX 1060 6GB

[ravikyram]

@ChowderII

I have tried in colab with TF nightly version (2.4.0-dev20200811) and i am seeing different error message.
TypeError: 'module' object is not callable
Please, find the gist here.Thanks!

[ChowderII]

Even if your error is different, it still implies there is something wrong with the load function right?
(I don't know if the tag awaiting response is for me or the tensorflow gods)

[saberkun]

I don't think we can always expect the customized subclass models be able to load with keras API.
Instead, could you first try the savedmodel API: https://www.tensorflow.org/api_docs/python/tf/saved_model/load?

[ChowderII]

I did try to load it that way, it gives me an AutoTrackable object that I can't seem to interact with the way I want. I want to save the as .h5 for another application but I get the same error message as my first post when I try to save it that way.
In the README of object detection, they state that their model for TF2 is built using keras, why can't we load it using keras too?

[ChowderII]

9026

@ravikyram
It seems this user is trying to do what I am trying to do as well. But we cannot perform tf.keras.models.load_model it crashes every time.

[ChowderII]

I'm very sorry to multi post but is there anything I can do to help you help me?
Let me know if my problem is not clear or needs clarification. Thank you!

[advaza]

I have a similar issue with a different object detection model (efficientdet/d0).

To reproduce:

1. Download efficientdet d0 model from model zoo - link
2. Run the following code.

My Environment is

• operating system: macOS, Catalina, 10.15.4
• tensorflow version: 2.3.1
• python version: 3.6

import tensorflow as tf
effdet0_path = "efficientdet_d0_coco17_tpu-32/saved_model"
keras_model = tf.keras.models.load_model(effdet0_path, compile=False)

Error:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-12-de1beb37d1c6> in <module>
----> 1 keras_model = tf.keras.models.load_model(effdet0_path, compile=False)

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile, options)
    185     if isinstance(filepath, six.string_types):
    186       loader_impl.parse_saved_model(filepath)
--> 187       return saved_model_load.load(filepath, compile, options)
    188
    189   raise IOError(

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in load(path, compile, options)
    119
    120   model = tf_load.load_internal(
--> 121       path, options=options, loader_cls=KerasObjectLoader)
    122
    123   # pylint: disable=protected-access

~/anaconda/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py in load_internal(export_dir, tags, options, loader_cls)
    631       try:
    632         loader = loader_cls(object_graph_proto, saved_model_proto, export_dir,
--> 633                             ckpt_options)
    634       except errors.NotFoundError as err:
    635         raise FileNotFoundError(

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in __init__(self, *args, **kwargs)
    192     self._models_to_reconstruct = []
    193
--> 194     super(KerasObjectLoader, self).__init__(*args, **kwargs)
    195
    196     # Now that the node object has been fully loaded, and the checkpoint has

~/anaconda/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py in __init__(self, object_graph_proto, saved_model_proto, export_dir, ckpt_options)
    128       self._concrete_functions[name] = _WrapperFunction(concrete_function)
    129
--> 130     self._load_all()
    131     self._restore_checkpoint()
    132

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _load_all(self)
    219
    220     # Finish setting up layers and models. See function docstring for more info.
--> 221     self._finalize_objects()
    222
    223   @property

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _finalize_objects(self)
    524         layers_revived_from_saved_model.append(node)
    525
--> 526     _finalize_saved_model_layers(layers_revived_from_saved_model)
    527     _finalize_config_layers(layers_revived_from_config)
    528

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _finalize_saved_model_layers(layers)
    704         call_fn = _get_keras_attr(layer).call_and_return_conditional_losses
    705         if call_fn.input_signature is None:
--> 706           inputs = infer_inputs_from_restored_call_function(call_fn)
    707         else:
    708           inputs = call_fn.input_signature[0]

~/anaconda/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/load.py in infer_inputs_from_restored_call_function(fn)
    980     return tensor_spec.TensorSpec(defun.common_shape(x.shape, y.shape),
    981                                   x.dtype, x.name)
--> 982   spec = fn.concrete_functions[0].structured_input_signature[0][0]
    983   for concrete in fn.concrete_functions[1:]:
    984     spec2 = concrete.structured_input_signature[0][0]

IndexError: list index out of range

[xumengwei]

@ChowderII have you solved this issue? I also want to load the OD models with keras API.

[ChowderII]

@ChowderII have you solved this issue? I also want to load the OD models with keras API.

I have not, no one is helping. It seems to be a problem with tensorflow itself. We need to wait until the assignees have time to look at this issue.

[xumengwei]

I've added a similar issue here but also no response yet... I feel it could be an issue of version diff about SavedModel format. Hope someone can help one this. I feel it's a common demand to load the OD models with keras format.