Serving: predict_pb2.py missing

[evolu8]

Having built from src I'm trying to run the inception example. It builds. The server run shows no errors. Then when I try to run the client I get an import error as there apears to be no predict_pb2.py

prediction_service_pb2 is there however.

python inception_client.py
Traceback (most recent call last):
  File "inception_client.py", line 26, in <module>
    from tensorflow_serving.apis import predict_pb2
ImportError: cannot import name predict_pb2

[kirilg]

You'll need to run the python binary from the bazel compiled directory instead so it can correctly find all dependencies. e.g. from the inception tutorial:
bazel-bin/tensorflow_serving/example/inception_client --server=localhost:9000 --image=/path/to/my_cat_image.jpg

[evolu8]

Thanks for the response! But this is a very awkward way of working. e.g. running it in pycharm debugger is a highly productive way to learn a project. The reliance on bazel compilation messes this up badly. I think bazel is an amazing build tool, but for scripting languages, it not only seems like overkill, but actually harms productivity a lot.

Sorry for the opinionated post, but I know I'm not alone, every developer i know working on TF projects via python have said similar things. is there any hope that the python will be disentangled from the bazel build process? Coupling via importable python libraries seems a simpler and more manageable interface for most people actually using the TF tools. I mean this to be constructive, not an ungrateful moan.

[kirilg]

I understand the frustration, and thanks for the feedback! I know TensorFlow lets you install their package directly and you can just import it in your Python code. In TensorFlow Serving, we have very little python (only in our /examples/ folder where we have sample export and client code), so providing such a package doesn't seem as critical.

Bazel puts some compilation artifacts and final outputs in the bazel-bin/ directory, and at least for python binaries, you can work with them as you would with regular code. For example if you look at the file bazel-bin/tensorflow_serving/example/inception_client, it itself is just a python boilerplate script that loads and runs "tf_serving/tensorflow_serving/example/inception_client.py" from your workspace. Can you run that script with a pycharm debugger? The same code should execute so it should hopefully work, but I've never done that myself.

Also note that you only need to compile it once to generate the bazel-bin/tensorflow_serving/example/inception_client run script. Since it just links in the real source file mentioned, any changes to your source file will automatically get picked up.

[vscv]

update: The problem is not solved. mnist_export/client is easy to bypass the error BUT the inception_export/client is NOT.

---

I have same problem.

*build
~/serving$ bazel build //tensorflow_serving/example:mnist_client
INFO: Found 1 target...
Target //tensorflow_serving/example:mnist_client up-to-date:
bazel-bin/tensorflow_serving/example/mnist_client
INFO: Elapsed time: 0.324s, Critical Path: 0.00s

[Bazel way]

• run client
  $bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000
  Traceback (most recent call last):
  File "/home/wei/serving/bazel-bin/tensorflow_serving/example/mnist_client.runfiles/tf_serving/tensorflow_serving/example/mnist_client.py", line 33, in
  from grpc.beta import implementations
  ImportError: No module named grpc.beta

Im sure the file is exist in /usr/local/lib/python3.4/dist-packages/grpc/beta/implementations.py
$ py
Python 3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.

from grpc.beta import implementations

It can be import from python3.

So, I
$vim bazel-bin/tensorflow_serving/example/mnist_client
...

!/usr/bin/env python3

PYTHON_BINARY = '/usr/bin/python3'
...
Change python to python3 and it works!!

$ bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000
Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Inference error rate: 13.100000000000001%

[Python way]
If I use
$py mnist_client.py --num_tests=100 --server=localhost:9000
Traceback (most recent call last):
File "mnist_client.py", line 37, in
from tensorflow_serving.apis import predict_pb2
ImportError: cannot import name 'predict_pb2'

Im sure the predict_pb2 is not exist in tensorflow_serving/apis/. It seems tensorflow_serving.apis missing 'predict_pb2.y' ?

It will be better if I can use [Python way] when client side without installation of JDK, Bazel... etc.

[tobegit3hub]

If you need model_pb2.py, predict_pb2.py or prediction_service_pb2.py, we have generated from protobuf files and you can download in deep_recommend_system.

This have been discussed for long time. If you're using bazel, everything should work like a charm. If you're using Python purely, we have the solution for this. Download the protobuf files from tensorflow/tensorflow and tensorflow/serving projects, use gRPC library to generate the file. You can use this script directly in https://github.com/tobegit3hub/deep_recommend_system/tree/master/python_predict_client/generate_python_files .

[tobegit3hub]

The predict_pb2.py is generated by bazel. I'm not sure if they will be added in the repo. So can my comment help to resolve your issue? @evolu8

[vscv]

Thanks for your sharing :)
In this issue I had added predict_pb2.py and others by clone the 'inception model' to the TF_serving. Still no luck.
However, I wii try your solution then report here.

[tsaiian]

@tobegit3hub Thank you!! it works perfectly for me.

[pakrchen]

@tobegit3hub Thank you so much!

[davidlenz]

worked for me, thank you @tobegit3hub !

[sumsuddin]

pip install tensorflow-serving-api

This command resolved the issue for me. Details are here.

[kirilg]

That's right, TensorFlow Serving now has a pip package as @sumsuddin mentioned that includes the mentioned proto files.

[jaredmessenger]

The pip install tensorflow-serving-api looks like it's just for python 2. Are there any plans to support python 3?

[wincle]

Worked for me ,Thx! @sumsuddin

[AnderUstarroz]

The serving api is also available for python 3:
pip install --upgrade tensorflow-serving-api-python3