Issue tracker for discussion with @jwmueller @cgraywang today.
results returned by fit may not be a good way to saving state and handling multiple experiments.
We decide to make a Predictor object which contains the following method:
save()
load()
predict()
evaluate()
I think this object should also support a fit_summary() method which returns something like the current Results object, with more informative/comprehensive instance-variable names as described here:
Also I'd make sure all task.fit() calls automatically call save at the end before returning. We do not want to burden the user with having to explicitly call save.
By the way, load() should be a class method, so the user can call task.load(filepath) without any predictor object.
In contrast, save() should be an instance method, so the user calls predictor.save(filepath).
Just to be clear, we should also try to ensure predictor.predict() is overloaded to be able to handle:
1) single example (e.g. single image, single document)
2) batch of many examples
3) MXNet Dataset object (ie. already preprocessed examples with a defined iterator)
For classification tasks, the sklearn convention is to use: predict_proba underneath the hood to produce predicted class probabilities, and then predict is just a wrapper around this that computes the argmax of the class probabilities.
Just to be clear, we should also try to ensure
predictor.predict()is overloaded to be able to handle:
- single example (e.g. single image, single document)
- batch of many examples
- MXNet Dataset object (ie. already preprocessed examples with a defined iterator)
For classification tasks, the sklearn convention is to use:
predict_probaunderneath the hood to produce predicted class probabilities, and thenpredictis just a wrapper around this that computes the argmax of the class probabilities.
This is great suggestion. I will work on it 馃憤