Serving: Support model download via INIT containers

[rakelkar]

In what area(s)?

/area API

/area autoscale
/area build
/area monitoring
/area networking
/area test-and-release

Describe the feature

KFServing would like to enable data scientists to refer to ML models in a consistent way across ML Inference Hosts (like TFServing and TensorRT) without requiring the build of a derived container.

See discussion and proposed design doc at: https://github.com/kubeflow/kfserving/issues/148

This requires KNative support for:

1. An INIT container to download the model before the inference container starts
2. The ability to mount a volume from the INIT container to the inference container

We believe that (2) above is a clean extension and does not violate the immutability principle laid out in issues like #2025

[rakelkar]

Not sure if area label is correct

[rakelkar]

@mattmoor

[knative-housekeeping-robot]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle stale

[mattmoor]

/remove-lifecycle stale

This also needs a place to put the model, so some volume work would be needed.

cc @savitaashture @markusthoemmes who are looking at multi-container, as I've mentioned these things as related work a few times.

[knative-housekeeping-robot]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle stale

[mattmoor]

/remove-lifecycle stale

[VonRehberg]

Would be good to have PVCs or any similar possibility to access models/files. Building derived containers seems very cumbersome.
Any feedback if you're working on it?

[mattmoor]

The most related thing in-flight is support for multiple spec.template.spec.containers, but I see spec.template.spec.initContainers, and expanded volume support as related (e.g. as a medium for passing data between containers). (cc @savitaashture @markusthoemmes )

I think at this point it is mostly a question of bandwidth. I know @JRBANCEL mentioned this on our call Weds, so maybe he is interested in tackling volumes next?

cc @dprotaso

[JRBANCEL]

I know @JRBANCEL mentioned this on our call Weds, so maybe he is interested in tackling volumes next?

Definitely, as this has been asked quite a bit already. Let's finish the Downward API work first though🚀 .

[knative-housekeeping-robot]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle stale

[mattmoor]

/remove-lifecycle stale

[mattmoor]

@JRBANCEL Any interest in tackling this for 0.17?

[pyramation]

so sounds like init-containers are in the works, or at least being looked at?

[mattmoor]

I think it still needs a driver, but it is something I'd love to see. 😇

[antongri]

Are there any news about this issue?

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.