Serving: How to export tf.learn.DNNClassifier model for serving?

[haiy]

I'v trained a dnn model like the example in here ,but I don't know the correct way to export the model for serving.

[martinwicke]

@davidsoergel has documentation in the works.

[haiy]

@martinwicke I've tried a way to export model but failed to deploy. I've already opened an issue #458 .I would appreciate that if you can help to fix it .Or would you please give some detail info about your doc url ?Thanks.

[haiy]

Any process info? @martinwicke

[martinwicke]

@davidsoergel got sidetracked, but I have seen a draft. It'll appear eventually.

[mvsusp]

I'm struggling with the same issue here.

[haiy]

@mvsusp I've resloved this .This was actually caused by a bug in protobuf https://github.com/google/protobuf/issues/3381. Now I'm serving models in a direct way.

save and load model:

def save_tf_learn_model(estimator, model_name, export_dir, feature_columns, ):
    feature_spec = create_feature_spec_for_parsing(feature_columns)
    serving_input_fn = input_fn_utils.build_parsing_serving_input_fn(feature_spec)
    export_dir = os.path.join(export_dir, model_name)
    estimator.export_savedmodel(export_dir, serving_input_fn)
    print("Done exporting tf.learn model to " + export_dir + "!")

SESS_DICT = {}
def get_session(model_id):
    global SESS_DICT
    config = tf.ConfigProto(allow_soft_placement=True)
    SESS_DICT[model_id] = tf.Session(config=config)
    return SESS_DICT[model_id]

def load_tf_model(model_path):
    sess = get_session(model_path)
    tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], model_path)
    return sess

now serving service get input and predict op:

input_x_holder =sess.graph.get_operation_by_name("input_example_tensor").outputs[0]
#check your dnn classifier txt pb to know which operation you should use.
predictions_holder = sess.graph.get_operation_by_name("dnn/binary_logistic_head/predictions/probabilities").outputs[0]

do predict

float_features = tf.train.Feature(float_list=tf.train.FloatList(value=refined_input))
feature_dict = {"": float_features}
example = tf.train.Example(features=tf.train.Features(feature=feature_dict))
serialized = example.SerializeToString()
 score = self.model.run([self.predictions_holder],
                               {self.input_x_holder: [serialized]})

Hope this help!

[mvsusp]

Thank you so much @haiy

I solved my issue by using tf.estimator.DNNClassifier instead of the tf.learn.DNNClassifier learn one.

here is my code:

INPUT_TENSOR_NAME = 'inputs'


def estimator(model_path):
    feature_columns = [tf.feature_column.numeric_column(INPUT_TENSOR_NAME, shape=[4])]
    return tf.estimator.DNNClassifier(feature_columns=feature_columns,
                                      hidden_units=[10, 20, 10],
                                      n_classes=3,
                                      model_dir=model_path)


def serving_input_receiver_fn():
    feature_spec = {INPUT_TENSOR_NAME: tf.FixedLenFeature(dtype=tf.float32, shape=[4])}
    return tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)()


def train_input_fn(training_dir):
    training_set = tf.contrib.learn.datasets.base.load_csv_with_header(
        filename=os.path.join(training_dir, 'iris_training.csv'),
        target_dtype=np.int,
        features_dtype=np.float32)

    return tf.estimator.inputs.numpy_input_fn(
        x={INPUT_TENSOR_NAME: np.array(training_set.data)},
        y=np.array(training_set.target),
        num_epochs=None,
        shuffle=True)()

[sukritiramesh]

For documentation related to using SavedModel with Estimators, see: https://www.tensorflow.org/versions/master/programmers_guide/saved_model#using_savedmodel_with_estimators.

[wjmaclean]

I'm trying to export a fairly simple tf.estimator.DNNClassifier from a saved_model, but I am hitting:

Unsupported Ops in the model
ParseExample

Any thoughts on why this is, or how to get around it?

[jorgeb49]

Hi @mvsusp. How are you querying the server? I am doing
curl -d '{"inputs":[1,2,3,4]}' -X POST http://localhost:8501/v1/models/iris_model:predict
But I get:
{ "error": "JSON Value: 1 Type: Number is not of expected type: string" }

My serving inout function is
def serving_input_fn(hyperparameters): feature_spec = {INPUT_TENSOR_NAME: tf.FixedLenFeature(dtype=tf.float32, shape=[4])} return tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)()

Where INPUT_TENSOR_NAME='inputs'

I also wonder why in the current example it says serving_input_fn while you use serving_input_receiver_fn ...

Thanks for any lights you can bring here.

[bwindsor22]

@wjmaclean thanks, new link: https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators