Serving: ImportError: No module named 'tensorflow_serving.apis.get_model_status_pb2' (api 1.7.0)

[cchung100m]

Dear all,

I try to use GetModelStatus() but encounter the import error issue, it seems that both tensorflow-serving-api 1.7.0 and 1.8.0 version don't have the related get_model_status_pb2 module

Is it a missing piece in tensorflow-serving-api?

mnist_client_query_model_version_v2.py code:

from __future__ import print_function
import sys
from grpc.beta import implementations
import tensorflow as tf
from tensorflow_serving.apis import model_service_pb2_grpc
from tensorflow_serving.apis import get_model_status_pb2
from tensorflow_serving.apis import get_model_metadata_pb2
from tensorflow_serving.apis import prediction_service_pb2
from tensorflow_serving.apis import predict_pb2, classification_pb2, inference_pb2, regression_pb2

tf.app.flags.DEFINE_string('server', '', 'PredictionService host:port')
FLAGS = tf.app.flags.FLAGS

def get_model_status(hostport):
        request = get_model_status_pb2.GetModelStatusRequest()
        request.model_spec.name = 'mnist'
        request.metadata_field.append("signature_def")

        host, port = hostport.split(':')
        channel = implementations.insecure_channel(host, int(port))
        stub = model_service_pb2_grpc.ModelServiceStub(channel)
        response = stub.GetModelStatus(request, 10)
        print(response.model_spec.version.value)

def main(_):
  if not FLAGS.server:
    print('please specify server host:port')
    return

  print('Sending GetModelStatus request...')
  get_model_status(FLAGS.server)

if __name__ == '__main__':
  tf.app.run()

ERROR MESSAGE:

root@59dbb35082c6:/notebooks/downloads# python3 mnist_client_query_model_version_v2.py --server=localhost:9000
/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Traceback (most recent call last):
  File "mnist_client_query_model_version_v2.py", line 23, in <module>
    from tensorflow_serving.apis import model_service_pb2_grpc
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_serving/apis/model_service_pb2_grpc.py", line 24, in <module>
    import tensorflow_serving.apis.get_model_status_pb2 as tensorflow__serving_dot_apis_dot_get__model__status__pb2
ImportError: No module named 'tensorflow_serving.apis.get_model_status_pb2'

THERE IS NO get_model_status API in Python package tensorflow-serving-api

root@59dbb35082c6:/downloads# ls -ltrh /usr/local/lib/python3.5/dist-packages/tensorflow_serving/apis/
total 108K
-rw-r--r-- 1 root staff    0 Jun 19 16:09 __init__.py
-rw-r--r-- 1 root staff 7.9K Jun 19 16:09 regression_pb2.py
-rw-r--r-- 1 root staff  18K Jun 19 16:09 prediction_service_pb2.py
-rw-r--r-- 1 root staff  10K Jun 19 16:09 predict_pb2.py
-rw-r--r-- 1 root staff 3.1K Jun 19 16:09 model_service_pb2_grpc.py
-rw-r--r-- 1 root staff 8.8K Jun 19 16:09 model_service_pb2.py
-rw-r--r-- 1 root staff 3.4K Jun 19 16:09 model_pb2.py
-rw-r--r-- 1 root staff 7.5K Jun 19 16:09 input_pb2.py
-rw-r--r-- 1 root staff  11K Jun 19 16:09 inference_pb2.py
-rw-r--r-- 1 root staff  12K Jun 19 16:09 get_model_metadata_pb2.py
-rw-r--r-- 1 root staff 9.8K Jun 19 16:09 classification_pb2.py
drwxr-sr-x 2 root staff 4.0K Jun 19 16:10 __pycache__

System Information:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

root@59dbb35082c6:/notebooks/downloads/examples# pip list | grep -i "tensorflow"
tensorflow             1.7.0
tensorflow-gpu         1.4.0
tensorflow-serving-api 1.7.0
tensorflow-tensorboard 0.4.0


root@59dbb35082c6:/downloads# apt list | grep -i "tensorflow"
tensorflow-model-server/unknown,now 1.8.0 all [installed]
tensorflow-model-server-universal/unknown 1.8.0 all


root@59dbb35082c6:/downloads# apt show tensorflow-model-server
Package: tensorflow-model-server
Version: 1.8.0
Built-Using: Bazel
Priority: optional
Section: contrib/devel
Maintainer: TensorFlow Serving team
Installed-Size: unknown
Homepage: https://github.com/tensorflow/serving
Download-Size: 98.0 MB
APT-Manual-Installed: yes
APT-Sources: http://storage.googleapis.com/tensorflow-serving-apt stable/tensorflow-model-server amd64 Packages
Description: TensorFlow Serving ModelServer

[AswathKiruba]

I am facing the same error.

[gautamvasudevan]

Will look into this

[gautamvasudevan]

This should be fixed in the next PyPi release. Thanks for reporting!

[cchung100m]

Hi @gautamvasudevan ,

I notice there is an update on API verion 1.7.1 and I download it to test whether the issue is solved or not

https://pypi.org/project/tensorflow-serving-api/1.7.1/#files

But I encounter another error response as below:

root@59dbb35082c6:/notebooks/downloads/tf_serving_exporter# python tf_serving_exporter.py
Traceback (most recent call last):
  File "tf_serving_exporter.py", line 11, in <module>
    from tensorflow_serving.apis import get_model_status_pb2, model_service_pb2
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_serving/apis/get_model_status_pb2.py", line 17, in <module>
    from tensorflow_serving.util import status_pb2 as tensorflow__serving_dot_util_dot_status__pb2
ImportError: No module named util

System information:

root@59dbb35082c6:/notebooks/downloads/installation_file# pip list | grep -i "tensorflow"
tensorflow             1.8.0
tensorflow-serving-api 1.7.1

[gautamvasudevan]

The next release will be 1.9.0 - should be out soon (corresponding with the next TensorFlow release).

[AswathKiruba]

@cchung100m Did you look into this, https://stackoverflow.com/questions/48113444/get-info-of-exposed-models-in-tensorflow-serving
Hope this helps you

  request = get_model_metadata_pb2.GetModelMetadataRequest()
  request.model_spec.name = 'your_model_name'
  request.metadata_field.append("signature_def")
  response = stub.GetModelMetadata(request, 10)

  print(response.model_spec.version.value)
  print(response.metadata['signature_def'])`

[cchung100m]

Hi @AswathKiruba

Thank you for the suggestion.
I had tried the api GetModelMetadata() but the response only return the latest version of model. What I want is Tensorflow Serving server return all serving models' metadata.

Thanks again,

[CHneger]

have you solve the problem. I have the requirement now, could you tell how you do it

[CHneger]

have you solve the problem. I have the requirement now, could you tell how you do it

@cchung100m

[AswathKiruba]

@cchung100m I was able to implement the getModelStatus(TFS 1.6.0) in Java. All you need to in do python is to get the compiled protobuf python files. Refer to this link on compiling the protobuf files https://developers.google.com/protocol-buffers/docs/pythontutorial

Also, you could load protobufs dynamically i.e. at the runtime and then create a prediction service like this:

this.tfServing = grpc.load(this.PROTO_PATH).tensorflow.serving;
this.client = new this.tfServing.PredictionService(this.tfServerUrl, grpc.credentials.createInsecure());

[cpdiku]

Using Tensorflow 1.9.0 and Tensorflow Serving API 1.9.0, the following code:
from tensorflow_serving.apis import get_model_status_pb2

gives me the error:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.5/dist-packages/tensorflow_serving/apis/get_model_status_pb2.py", line 17, in <module> from tensorflow_serving.util import status_pb2 as tensorflow__serving_dot_util_dot_status__pb2 ImportError: No module named 'tensorflow_serving.util'

[hwaking]

I'm using tf serving in centos system with python 3.6 and tensorflow 1.10.0 and got the problem "ImportError: No module named tensorflow_serving.apis"

[BIG-CHENG]

In my cases, just simply install it via pip, e.g.
"pip install tensorflow-serving-api"
solve that problem

[ebertone]

In my cases, just simply install it via pip, e.g.
"pip install tensorflow-serving-api"
solve that problem

Thanks!

[rmothukuru]

In my cases, just simply install it via pip, e.g.
"pip install tensorflow-serving-api"
solve that problem

Thank you

[NOSHEENSANAY]

thanks