Serving: Serving Python Import Error

[ghost]

I built the Tensorflow Serving package on an Ubuntu AWS Machine and it passed all of the tests. However, when I import the package, I receive the following error:

ubuntu@ip-172-31-5-123:~/serving/tensorflow_serving/example$ python mnist_export.py
Traceback (most recent call last):
  File "mnist_export.py", line 32, in <module>
    from tensorflow_serving.example import mnist_input_data
ImportError: No module named tensorflow_serving.example

When I run

bazel test tensorflow_serving/...

I shows me that all of the tests have passed.

//tensorflow_serving/util:any_ptr_test                          (cached) PASSED in 0.0s
//tensorflow_serving/util:cleanup_test                          (cached) PASSED in 0.1s
//tensorflow_serving/util:event_bus_test                        (cached) PASSED in 0.1s
//tensorflow_serving/util:fast_read_dynamic_ptr_benchmark       (cached) PASSED in 28.0s
//tensorflow_serving/util:fast_read_dynamic_ptr_test            (cached) PASSED in 0.1s
//tensorflow_serving/util:inline_executor_test                  (cached) PASSED in 0.0s
//tensorflow_serving/util:observer_test                         (cached) PASSED in 0.2s
//tensorflow_serving/util:optional_test                         (cached) PASSED in 0.1s
//tensorflow_serving/util:periodic_function_test                (cached) PASSED in 0.5s
//tensorflow_serving/util:threadpool_executor_test              (cached) PASSED in 0.1s
//tensorflow_serving/util:unique_ptr_with_deps_test             (cached) PASSED in 0.0s

Executed 0 out of 44 tests: 44 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

In any case, would love any insights you may have.

[fangweili]

Short answer:

You should build the target with:
ubuntu@ip-172-31-5-123:~/serving$ bazel build tensorflow_serving/example:mnist_export
And run it with:
ubuntu@ip-172-31-5-123:~/serving$ bazel-bin/tensorflow_serving/mnist_export

Long answer:

Tensorflow Serving does not currently have binary release. Therefore you need to build against its source from scratch for development. For bazel-based project, the build output are organized as this. For python binary target in particular, you have to execute by running the generated shell script instead -- in this case mnist_export. The script sets up proper package/library search path -- to find tensorflow_serving.example package, before executing mnist_export.py.

[fangweili]

closing... feel free to reopen if there's still questions.

[coltsfan]

Hello fangweili: Thanks for the response. I understand your point at a high level but I still don't know how you would do that tactically. Can you elaborate on the steps I would need to take in order to import the tesnorflow_serving module in raw python? I apologize for asking such a trivial question -- for the novices amongst us, that documentation would be super useful.

[fangweili]

Do you want to just deploy the mnist example on AWS? Or are you trying to use Tensorflow Serving Exporter in your python code?

[coltsfan]

I want to use TFlow Serving Exporter in my Python Code.

[vodp]

This means tensorflow_serving cannot be used in an interactive python command line?

[sukritiramesh]

Hi @vodp, based on @fangweili's response earlier, I think that is correct. Given that there is no release currently, in the context of export.py, this would probably result in a failed import on the tensorflow_serving package.

[suraj-vantigodi]

@fangweili
I want to deploy machine translation example on AWS.
Currently I have created a REST webservice. But here the problem is the model and the vocabulary files get loaded for every hit, so its time consuming. That is when I came across Tensorflow serving.
Can you please give me some inputs on how to go ahead. The MINST example is not that clear.

Also when building the test (bazel test tensorflow_serving/...) I get the output as Executed 0 out of 50 tests : 1 fails to build and 49 were skipped.

[blusa]

You can work it around writing your export code within the scope of tensorflow_serving, adding it to bazel and compiling again

[kirilg]

@suraj1990 Did you get a chance to go through https://tensorflow.github.io/serving/ and the architecture overview/tutorials? Can you elaborate on what part was unclear?

Did you have any local changes when running bazel test? The build is not broken so everything should run. Please include more details about the failed build including the output errors if it doesn't include your local changes (probably best to do that in a separate github issue).

[kirilg]

Following up on the original question: The exporter (along with the rest of session_bundle) were moved to Tensorflow/contrib in the main Tensorflow repo, so the exporter is now part of the regular release. That means that you can either link it in directly and compile using Bazel (as mentioned before), or install Tensorflow using the pip package and import directly in your Python program as you would any other Tensorflow python code.

[pauldb89]

I replaced
from tensorflow_serving.example import mnist_input_data
with
from tensorflow.contrib.learn.python.learn.datasets import mnist as mnist_input_data
and it worked for me.

[ronilp]

@pauldb89 how did you know tensorflow.contrib.learn.python.learn.datasets should come in place of tensorflow_serving.example ?