Kfserving: Inference for deployed triton pod fails, is there a way to determine why?

[mokpolar]

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

I downloaded "mobilenet" and saved it as a pb file, created config.pbtxt and deployed it.

config.pbtxt

name: "my_model"
platform: "tensorflow_savedmodel"
max_batch_size: 1
input [{
    name: "input_1"
    data_type: TYPE_FP32
    dims: [ 224, 224, 3 ]
  }
]
output [{
    name: "act_softmax"
    data_type: TYPE_FP32
    dims: [ 1000 ]
  }
]
instance_group [
    {
        count: 1
        kind: KIND_CPU
    }
]

And after making the image into json file in 224,224,3 form, executed the inference.
And I got 400 Bad Request error.

I want to know where the problem occurred, but like other inference pods, there is no reason for the failure in the log.
Is there any way to find out why?

What did you expect to happen:
image prediction.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Istio Version:
• Knative Version:
• KFServing Version: 0.4.0
• Kubeflow version:
• Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
• Minikube version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue.
See dashboard for more details.

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue.
See dashboard for more details.

[yuzisun]

@deadeyegoodwin Any idea?

[deadeyegoodwin]

The 400 response should return a json body with error details, if the error is coming from Triton. You can also run Triton with --log-verbose=1 and check the container logs to see if there is some console output indicating reason for failure.

[mokpolar]

Sorry, I tried without understanding the structure of the Triton inference.
Like any other inferenceservices, the error was caused by trying to request POST to the inferenceservice.

As in the BERT example, I created a transformer and used InferContext for the predict function.
I got

{"predictions": [["plastic_bag", "27.64%"]]}

Thank you!!

[yuzisun]

@mokpolar Glad it is working for you! I am closing out this.