Kfserving: Tensorflow example not working

[Anderssorby]

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]
I tried to apply the config described in https://github.com/kubeflow/kfserving/tree/master/docs/samples/tensorflow

So I tried this config:

apiVersion: "serving.kubeflow.org/v1alpha2"
kind: "InferenceService"
metadata:
  name: "flowers-sample"
spec:
  default:
    predictor:
      tensorflow:
        storageUri: "gs://kfserving-samples/models/tensorflow/flowers"

And running kubectl apply -f tensorflow.yaml

When running kubectl get inferenceservice I get:

$ kubectl get inferenceservice
NAME                   URL   READY   DEFAULT TRAFFIC   CANARY TRAFFIC   AGE
flowers-sample               False                                      61s

I'm getting this when running kubectl get all

service.serving.knative.dev/power-price-forecast-predictor-default   http://power-price-forecast-predictor-default.kubeflow.example.com   power-price-forecast-predictor-default-89gcr                 Unknown   RevisionMissing

This URL is not going to work so how can I change it to something that will?

What did you expect to happen:

Get a working endpoint to access over HTTP.
When I run

MODEL_NAME=flowers-sample
INPUT_PATH=@./input.json
CLUSTER_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
SERVICE_HOSTNAME=$(kubectl get inferenceservice ${MODEL_NAME} -o jsonpath='{.status.url}' | cut -d "/" -f 3)

curl -v -H "Host: ${SERVICE_HOSTNAME}" http://$CLUSTER_IP/v1/models/$MODEL_NAME:predict -d $INPUT_PATH

Both CLUSTER_IP and SERVICE_HOSTNAME are empty.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:
(How do I get these versions?)

• Istio Version:
• Knative Version:
• KFServing Version:
• Kubeflow version:
• Minikube version:
• Kubernetes version: (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.10-dispatcher", GitCommit:"f5757a1dee5a89cc5e29cd7159076648bf21a02b", GitTreeState:"clean", BuildDate:"2020-02-06T03:29:33Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-gke.33", GitCommit:"2c6d0ee462cee7609113bf9e175c107599d5213f", GitTreeState:"clean", BuildDate:"2020-01-15T17:47:46Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}
• OS (e.g. from /etc/os-release): Arch Linux

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the labels:

| Label | Probability |
| ------------- | ------------- |
| bug | 0.98 |

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[yuzisun]

@Anderssorby can you get more detailed output with following command ?

Kubectl get ksvc power-price-forecast-predictor-default -oyaml

[Anderssorby]

Here:

apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
  annotations:
    serving.knative.dev/creator: system:serviceaccount:kubeflow:default
    serving.knative.dev/lastModifier: system:serviceaccount:kubeflow:default
  creationTimestamp: "2020-02-21T10:16:22Z"
  generation: 1
  name: power-price-forecast-predictor-default
  namespace: kubeflow
  ownerReferences:
  - apiVersion: serving.kubeflow.org/v1alpha2
    blockOwnerDeletion: true
    controller: true
    kind: InferenceService
    name: power-price-forecast
    uid: 3597ddbb-5493-11ea-a0c3-42010af00141
  resourceVersion: "586104"
  selfLink: /apis/serving.knative.dev/v1alpha1/namespaces/kubeflow/services/power-price-forecast-predictor-default
  uid: 359a0127-5493-11ea-a0c3-42010af00141
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/target: "1"
        internal.serving.kubeflow.org/storage-initializer-sourceuri: gs://flex-ml-models/test.pb
        queue.sidecar.serving.knative.dev/resourcePercentage: "0.2"
      creationTimestamp: null
      labels:
        serving.kubeflow.org/inferenceservice: power-price-forecast
    spec:
      containers:
      - args:
        - --port=9000
        - --rest_api_port=8080
        - --model_name=power-price-forecast
        - --model_base_path=/mnt/models
        command:
        - /usr/bin/tensorflow_model_server
        image: 'tensorflow/serving:'
        name: kfserving-container
        readinessProbe:
          successThreshold: 1
          tcpSocket:
            port: 0
        resources: {}
      timeoutSeconds: 60
  traffic:
  - latestRevision: true
    percent: 100
status:
  conditions:
  - lastTransitionTime: "2020-02-21T10:16:22Z"
    status: Unknown
    type: ConfigurationsReady
  - lastTransitionTime: "2020-02-21T10:16:22Z"
    message: Configuration "power-price-forecast-predictor-default" is waiting for
      a Revision to become ready.
    reason: RevisionMissing
    status: Unknown
    type: Ready
  - lastTransitionTime: "2020-02-21T10:16:22Z"
    message: Configuration "power-price-forecast-predictor-default" is waiting for
      a Revision to become ready.
    reason: RevisionMissing
    status: Unknown
    type: RoutesReady
  latestCreatedRevisionName: power-price-forecast-predictor-default-qfhwj
  observedGeneration: 1
  url: http://power-price-forecast-predictor-default.kubeflow.example.com

I got more hang of it now and I got it working with Tensorflow Serving. Possibly coming back to this project later though.

[yuzisun]

@Anderssorby I noticed that you deployed the service to kubeflow namespace, that won’t work because it is system namespace and service deployed there does not call kfserving pod mutator. If you deploy to default namespace or other user namespaces that should work.

[yuzisun]

/close

Please check docs here https://github.com/kubeflow/kfserving/blob/master/README.md#kfserving-in-kubeflow-installation.

[k8s-ci-robot]

@yuzisun: Closing this issue.

In response to this:

/close

Please check docs here https://github.com/kubeflow/kfserving/blob/master/README.md#kfserving-in-kubeflow-installation.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.