Mlj.jl: Feature importance / model inspection

[baggepinnen]

It would be nice to have some integrated tools for model inspection and feature importance (FI). Below are some links to resources and what's available in scikit learn.

Scikit learn exposes a number of tools for understanding the relative importance of features in a dataset. These tools are general in the sense that they can be made to work with many different kinds of models. They are organized in a module called "Inspection" which I find fitting, since they all allow the user to somehow understand or inspect the result of fitting a model in other ways than simply measuring the error/accuracy. Some of them are linked below

• Permutation FI in scikit. This one is very general, randomly shuffle one column (feature) and see how much worse the model performns.
• random forest FI in scikit
• partial dependence plot this is not really a FI tool, but nevertheless an interesting tool to have to understand the result of the fitted model.
• Drop Column FI This one is simple, train a model without a feature and see how much worse it gets.
• LIME A general and popular method model-agnostic tool for understanding the impact of perturbing inputs.
• A Unified Approach to Interpreting Model Predictions. A recent paper on model inspection, going through LIME and other related methods.

[tlienart]

Thanks for this.

For bagging ensembles it's reasonably straightforward. Some models we interface with also have it (e.g. XGBoost) and so it can just be part of the interface (via report), I'll have a look into this for xgb.

Support for permutation/drop FI seems reasonably easy, there's just a question as to _where_ the implementation would go, maybe a comparable "model(s) inspection" module or package in MLJ or something of the sorts.

The rest of your suggestions are a bit trickier. LIME is nice but basically is an entire package; like shap which is quoted in the article at your last point.

[ablaom]

Feature importance is an interesting one because most of the measures out there are rather ad-hoc and model dependent. That is, the very definition of feature importance depends on the model (eg, absolute value of a coefficient in a linear model makes no sense for a decision tree). And for certain models, eg trees and random forests, there are several inequivalent methods in common use. The paper cited above on shap describes an approach that is really model independent; unless someone is aware of another such approach, I suggest any generic MLJ tool follow that approach. There is already some implementation of SHAP in python, if I remember correctly.

[tlienart]

The recently created https://github.com/nredell/ShapML.jl may also be a very nice add (already compatible with MLJ as far as I can see) cc @nredell

[nredell]

My plans for ShapML can be found on the Discourse site--https://discourse.julialang.org/t/ml-feature-importance-in-julia/17196/12--, but I'm posting here for posterity sake.

Just sitting down for the first refactor/feature additions today. I'll code with these guidelines in mind (https://github.com/invenia/BlueStyle) as well as take a trip through the MLJ code base. And if a general feature importance package pops up in the future, I wouldn't be opposed to helping fold ShapML in if it's up to par and hasn't expanded too much by then.

[ablaom]

cc @sjvollmer (for summer FAIRness student, if not already aware)

[ablaom]

My current inclination is to see if this can be satisfactorily addressed with third party packages, such as the Shapley one. A POC would make a great MLJTutorial.

If something more integrated makes sense, though, l'm interested to here about it.