Spark-on-k8s-operator: Start looking at adding online model serving on the otherside

[holdenk]

Batch-only-training is cool, but do you know whats cooler? E2E training to serving!

We could look at the ML pipeline stage as the starting point. But maybe add trait for exportable to.

[holdenk]

So some initial notes on how we could consider approaching handling the dataprep / model serving:

• Export to a TF like format, reuse the existing TF serving container. Requires work for each stage.
• Leverage something like prediction.io
• Add support for PMML, would require adding PMML export for many of the stages which don't have it yet
• Hope another format / serving layer shows up (maybe PFA finally works?)
• Use spark-streaming to do serving (especially given the new execution engine options in 2.3+)
• Sub-class to get at the individual element transform functions and rewrite where necessary. Probably wrap in play or similar for serving, use Spark's built in object persistence layer

For reference see https://www.slideshare.net/databricks/how-to-productionize-your-machine-learning-models-using-apache-spark-mllib-2x-with-richard-garris & https://predictionio.apache.org/

Likely the simplest would be starting with predictionio and going from there. We could also support multiple serving options (e.g. simple things we might write support to exporting to TF but more complex depend on predictionio)

[skonto]

@holdenk how do you envision operator helping with this? Adding capabilities like model monitoring etc, extending sparkctl?