Serving: Bad request when serving mnist and making REST API

[Saxenauts]

Bug Report

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
• TensorFlow Serving installed from (source or binary): Docker and apt package
• TensorFlow Serving version: 1.13.0-rc1+dev.sha.f16e777

Describe the problem

Keras exported Saved Model not able to call REST API. I have a standard cnn model for MNIST dataset written in keras. The model works fine on keras. I exported that model to saved model with simple save. I can load and serve that model through tensorflow_model_server tool. On my system as well as the docker image.

When I call a non image based model with numerical data, it works fine. (ex. half_plus_two). I am able to call the REST API successfully. However if I replace the model with mnist and try to make a prediction request like this :

img = image.img_to_array(image.load_img('img_4.jpg', target_size=(28, 28)))/2$
img = img.astype('float16')

jpeg_bytes = base64.b64encode(data).decode()
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes

I see this error:

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:8501/v1/models/mnist_cnn:predict

And there is no response on the tf serving either. Atleast when I make a prediction request on this model with invalid data, I get a response error that says invalid input dimensions.

Also, exploring the model

 >> saved_model_cli show --dir . --tag_set serve --signature_def serving_default

The given SavedModel SignatureDef contains the following input(s):
  inputs['input_image'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 28, 28, 1)
      name: conv2d_1_input:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['activation_1/Softmax:0'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 10)
      name: activation_1/Softmax:0
Method name is: tensorflow/serving/predict

I have also tried

  predict_request = { "instances": [{'inside_image':img.tolist()}]}

Still gives the same bad request error.

I'm sure I am missing something trivial here. Please help me figure this out. Let me know if you need more info from me.

Test Image is a 28x28 image from the mnist dataset.

[netfs]

your mnist model requires a (-1, 28, 28, 1) dimensional tensor. the first dimension captures the
batch size (# images), so ignoring that -- you need a 28x28x1 float tensor that represents an image.

in your sample code, you are passing raw jpeg bytes. this only works if the model is capable of handling jpeg bytes directly (and internally decoding and converting it to required multi-dimensional float tensors). mnist model does not seem to have that (resnet has it). you can update (PR welcome ;-) mnist model to accept jpeg image bytes to get your current request to work.

to get existing mnist model to work, convert your image to a 28x28x1 float and use that in your request. something like this (untested):

#!/usr/bin/python

import numpy as np
from PIL import Image
import requests

def main():
  image_file = 'img_4.jpg'
  # Convert arbitrary sized jpeg image to 28x28 b/w image.
  data = np.array(Image.open(image_file).convert('L').resize((28, 28))).astype(np.float).reshape(-1, 28, 28, 1)

  # Dump jpeg image bytes as 28x28x1 tensor
  np.set_printoptions(threshold=np.inf)       
  json_request = '{{ "instances" : {} }}'.format(np.array2string(data, separator=',', formatter={'float':lambda x: "%.1f" % x}))
  resp = requests.post('http://localhost:8501', data=json_request)
  print('response.status_code: {}'.format(resp.status_code))     
  print('response.content: {}'.format(resp.content))

main()

clearly this yields a very verbose request. passing jpeg image bytes directly will be far better -- but will need minor changes to mnist model to accept jpeg bytes.

[netfs]

closing this issue, as the problem was in request creation.