Serving: Documentation for how estimator/graph_transformations/tf-serving work together

[gobrewers14]

System information

• TensorFlow Serving installed from binary
• TensorFlow Serving version: 1.10.0

Describe the Problem

The documentation describing how create and serve a custom Estimator, how to serve a tensorflow model in general, and how to perform graph transforms is very helpful - each on its own - but it is unclear how all these components fit together in the same ecosystem. This google cloud documentation on deploying models seems to suggest this (create -> transform -> serve) is the intent, I just cannot seem to find any documentation on how to:

• create a custom estimator
• perform graph transformations on the created model
• create a servable from the transformed model
• serve model

[Harshini-Gadige]

@gautamvasudevan - Hi, any update on this documentation ?

[gautamvasudevan]

No. @lamberta any thoughts on this?

[lamberta]

I've seen an internal doc that addresses some of this (b/116674557), but nothing that is ready to publish.
Might make a nice tutorial for the serving docs.

[r-wheeler]

Looking for this as well.

[r-wheeler]

@lamberta Anything on this you could share? The ideal scenario would be documentation on how to:

• create a custom estimator
• perform graph transformations on the created model
• create a servable from the transformed model
• serve model

I am able to accomplish many of these steps independently:

• Familiar with estimator api and writing custom model_fn etc
• able to perform graph transforms on non estimator based graphs
• able to create servables from both estimators and using the lower level apis

but pruning / transformer the output of an estimator, and generating a valid servable from this, would be great.

[lamberta]

The doc I saw became this blog post: Optimizing TensorFlow Models for Serving

We're building out the TFX section which will have more pipelines for serving: https://www.tensorflow.org/tfx/
For TFX, you can see the Chicago Taxi end-to-end example (and notebook).
We'll continue to build this out.

[Harshini-Gadige]

Thank you @lamberta.

[r-wheeler]

@lamberta thanks for posting