Mlj.jl: Add default_measure to learning_curve!

[tlienart]

In learning_curve! (but not only there) you can specify a measure which, by default, is set to rms; however if you leave it to default and try to do this on a classifier you'll get an error.

It would be easy to change so that it takes a reasonable default measure for the type of model something like

function learning_curve!(mach::Machine{<:Supervised};
                         resolution=30, resampling=Holdout(),
                         measure=default_measure(mach), operation=predict,
                         range=nothing, verbosity=1, n=1)

where default_measure returns for instance either rms or cross_entropy or some such based on what type of input it gets

[ablaom]

Actually, there is already a default_measure(model) function, defined in MLJBase/src/measures.jl, which uses the scientific type of the target to determine a sensible default. So,

function learning_curve!(mach::Machine{<:Supervised};
                         resolution=30, resampling=Holdout(),
                         measure=default_measure(mach.model), operation=predict,
                         range=nothing, verbosity=1, n=1)

ought to do it.

[Sinansi]

Hello,

Can someone confirm whether the MLJ.learning_curve! function has been deprecated?
I cannot find this function from MLJ documentation 0.3.0 onward.
So I checked 0.3.0 release notes, but couldn't find a deprecation announcement.

Thank you!

[ablaom]

Yes, good catch, thanks. It is not in the manual, will address this.

https://github.com/alan-turing-institute/MLJ.jl/issues/362

There is a docstring, so you can do ?learning_curve! at the REPL. And there is a demo at https://alan-turing-institute.github.io/MLJTutorials/pub/getting-started/ensembles-2.html

[Sinansi]

Thank you so much Dr. Anthony!
Yes, I have been working on the Tutorials since you provided me with the links.
Your tutorials are so informative and helpful, and I am glad they are available in various formats.
Great work!

[ablaom]

Your welcome. Kudos for the tutorials goes in large part to @tlienart.

[ablaom]

Resolved by https://github.com/alan-turing-institute/MLJ.jl/issues/358