Kfserving: Transfer context between preprocess and postprocess

[valentin-stefan-popa]

I need to transfer some infos from preprocess to postprocess.
For example if I pass an image to preprocess and I vectorize it and then I do a prediction at that image but at postprocess I need to create another image with same size but painting it base on prediction. For this I need some inputs like image size, shape, etc from preprocessing part.
Other example is that maybe in preprocess I pass a 3D image and in postprocess maybe I want to return a 2D image and for this I need again some infos from preprocess.

How I can pass now infos between preprocess and postprocess?

I think this is not possible now but looking in code here: https://github.com/kubeflow/kfserving/blob/master/python/kfserving/kfserving/handlers/http.py#L48 I can see this
request = model.preprocess(body) request = self.validate(request) response = model.predict(request) response = model.postprocess(response)
so there we can have one variable for transporting data from preprocess to postprocess something like this:
transport, request = model.preprocess(body) request = self.validate(request) response = model.predict(request) response = model.postprocess(transport, response)

Any better idea how we can safe transport things from preprocess to postprocess?

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the labels:

| Label | Probability |
| ------------- | ------------- |
| feature | 0.60 |

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[pugangxa]

Interesting case. I think you should already extends KFModel(say ExtendedModel) to have your own preprocess and postprocess right? Do you mind to have member variables in your ExtendedModel? So that you can set the value of the member variable in your preprocess function and use it directly in your postprocess function.

[valentin-stefan-popa]

I found a way to pass additional informations between preprocess and postprocess. Starting from this example :https://github.com/kubeflow/kfserving/blob/master/docs/samples/transformer/image_transformer/image_transformer/image_transformer.py#L41 I added in that class also predict method and ovverride original one like this:

def predict(self, request: Dict) -> Dict:
    response = super().predict(request)
    additional_info = request['additional_info']
    response['additional_info'] = additional_info
    return response

As you can see there the idea was to intercept predict response and add on it also my additional infos passed from preprocess. Like this we don't need to do any change on kfserving :)