Cuml: [FEA] Model save / load

[cjnolet]

It would be very useful if we had a mechanism on the Python side for saving/loading models that have been created with cuML.

It would be ideal if models could be serialized, like in sklearn. Even thought we aim to support "speed of light", naturally reducing the amount of time spent building models, it would be of great benefit to users to be able to store and recall models.

[dantegd]

Untested so far, but we should be able to pickle and load the models in a similar fashion to scikit-learn. Will test and update the issue if there are required changes needed to support this properly,

[cjnolet]

I have not tested this. The thought crossed my mind while looking at issues in the UMAP-learn github repository.

There doesn't seem to be a whole lot of concrete standardization in how we have structured our final cython classes and I'm working under the assumption that 1) pickling the uintptr_t objects is not going to work, and 2) pickling the device memory is not going to work.

Definitely good to try it out, though. If it does work, we have a new feature!

[taureandyernv]

Tested and it fails. We have no good way of accessing, saving, and loading the model - only the dataframe. I have verified that the fit returns a none type, and the variable containing the model is encapsulated. Can we please make this a feature?

[JohnZed]

Minimally need to document clearly how to do this, including what models do not save successfully.

[JohnZed]

Follow on: make KNN save-able

[JohnZed]

TODO for 0.8 - clearly document which models are picklabe. New doc page?

[adriantorrie]

Any chance of baking in native support to serialise to the ONNX format (ONNX-ML variant)?

This would allow:

• Quick productionising of models using the onnxruntime and served with docker via the onnx-server image/container.
• Integration with tools such as MLFLow to manage the machine learning lifecycle (it directly supports ONNX)
• ONNX is avaiable in conda-forge. I mention this because I note RAPIDS is using conda as their primary release strategy, and not supporting pip at this time

Note: Scikit-learn doesn't provide native support for exporting to the ONNX format, it requires the use of sklearn-onnx (which appears to succeed onnxmltools), so this could be an alternative path to serialising natively in ONNX format as well, if taken into consideration during your design phase.

Example of scikit-learn workflow for serialising to ONNX (from the docs):

# Train a model.
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y)
clr = RandomForestClassifier()
clr.fit(X_train, y_train)

# Convert into ONNX format with onnxmltools
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
initial_type = [('float_input', FloatTensorType([1, 4]))]
onx = convert_sklearn(clr, initial_types=initial_type)
with open("rf_iris.onnx", "wb") as f:
f.write(onx.SerializeToString())

# Compute the prediction with ONNX Runtime
import onnxruntime as rt
import numpy
sess = rt.InferenceSession("rf_iris.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]

Not sure if relevant: TensorRT has a backend for ONNX, just mentioning as I'm aware RAPIDS has some affiliation with Nvidia, perhaps there's something there that could possibly be leveraged?

[JohnZed]

Hi, @adriantorrie - TensorRT can parse ONNX for deep learning models. We haven't worked on native ONNX integration, but it would be cool to leverage the sklearn-onnx project if it's workable. I'm not sure how sklearn-onnx works internally, but if it queries sklearn models via public apis to get details, it may be pretty easy to bridge to cuml, since we follow the same apis.

[JohnZed]

TODO for 0.8 - do we need a notebook example for save/load via pickle? Or is just the test good enough?

[cjnolet]

@taureandyernv, I noticed you are working on one.

[andrewssobral]

Hello guys,
Following this closed thread, do you know where I can find a reference for cuML model save / load using Pickle or Joblib?
Thanks in advance

[andrewssobral]

I think I found what I was looking for:
https://github.com/rapidsai/cuml/blob/master/python/cuml/test/test_pickle.py