Serving: Export custom model with TensorFlow Serving

[TonTonTWMAN]

Hi, All

I have exported inception model with this link and these 2 following command::
https://tensorflow.github.io/serving/serving_inception
bazel build //tensorflow_serving/example:inception_export
bazel-bin/tensorflow_serving/example/inception_export --checkpoint_dir=inception-v3 --export_dir=inception-export

However, when I tried replace the "model.ckpt-157585" with trained model by myself, I have got this error; could someone help me to reslove this error ? I just try to export my custom model and deploy withtensorflow serving.

Thank you very much. !

`WARNING:tensorflow:From /serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py:94: index_to_string (from tensorflow.contrib.lookup.lookup_ops) is deprecated and will be removed after 2017-01-07.
Instructions for updating:
This op will be removed after the deprecation date. Please switch to index_to_string_table_from_tensor and call the lookup method of the returned table.
Traceback (most recent call last):
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 169, in
tf.app.run()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 165, in main
export()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 105, in export
saver.restore(sess, ckpt.model_checkpoint_path)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 1439, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [3] rhs shape= [1001]
[[Node: save/Assign_30 = Assign[T=DT_FLOAT, _class=["loc:@logits/logits/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](logits/logits/biases, save/RestoreV2_30)]]

Caused by op u'save/Assign_30', defined at:
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 169, in
tf.app.run()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 165, in main
export()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 100, in export
saver = tf.train.Saver(variables_to_restore)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 1051, in __init__
self.build()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 1081, in build
restore_sequentially=self._restore_sequentially)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 675, in build
restore_sequentially, reshape)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 414, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/training/saver.py", line 155, in restore
self.op.get_shape().is_fully_defined())
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
use_locking=use_locking, name=name)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/framework/ops.py", line 2389, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/framework/ops.py", line 1264, in __init__
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [3] rhs shape= [1001]
[[Node: save/Assign_30 = Assign[T=DT_FLOAT, _class=["loc:@logits/logits/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](logits/logits/biases, save/RestoreV2_30)]]`

[kirilg]

It looks like the two version are incompatible, and it's hard to investigate without knowing what the new model you're using is. If you're using your own version of inception that you modified, you'll probably just want to add exporting to your python script directly instead of saving a checkpoint, then loading that checkpoint in inception_export just to export. Basically just see what the exporter is used for in inception_export and do the same in your file.

If I had to guess, and it's a long shot without knowing what you changed, is your trained model using only 2 classes by any chance? The original is using 1000, so that would explain the error:
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [3] rhs shape= [1001]
(tensor size is num_classes+1)

[sskgit]

@ideex2 were you able to export custom model with Tensorflow serving? If so, would you please share how did you do it?

[gyang274]

@ideex2 It seems your customized model modified the graph by replacing the last layer to 3 classes, as stated by @kirilg.

From my experience, there are 2 ways often used to export a model for tensorflow serving.

A simple way is to export directly using SavedModel (https://www.tensorflow.org/programmers_guide/saved_model):

export_dir = ...
...
builder = tf.saved_model.builder.SavedModelBuilder(export_dir)
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph_and_variables(sess,
                                       [tag_constants.TRAINING],
                                       signature_def_map=foo_signatures,
                                       assets_collection=foo_assets,
                                       strip_default_attrs=True)
...
# Add a second MetaGraphDef for inference.
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph([tag_constants.SERVING], strip_default_attrs=True)
...
builder.save()

The other way is to customize the BUILD file to build the model using tensorflow serving, e.g., https://gyang274.github.io/tensorflow-serving-tutorial/0x02b00.slim.inception.v4.html.

[ymodak]

@ideex2 Is this still an issue?

[ymodak]

Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!