Shap: Exception: Model type not yet supported by TreeExplainer: <class 'sklearn.pipeline.Pipeline'>

[Sajenthan]

I'm getting the following error when i executed the TreeExplainer, could you help on this.

clf = Pipeline(steps=[('preprocessor', transformations),
('classifier', RandomForestRegressor(max_depth=6, random_state=0))])

model = clf.fit(X_train, Y_train)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_train)

Exception: Model type not yet supported by TreeExplainer:

[erickrf]

Your model is actually a pipeline (usually a sequence of preprocessing operations in sklearn followed by the actual classifier).
It seems you'll have to do the preprocessing by hand and only give the classifier itself to the explainer.

[slundberg]

@erickrf is right. If you need to explain a whole pipeline I recommend using a model agnostic approach, which under the new API can be explainer = shap.Explainer(f, X); explainer(X)

[Sajenthan]

Thanks @erickrf & @slundberg , I'm getting the following error

Exception Traceback (most recent call last)
in
----> 1 explainer = shap.Explainer(model, X_train)
2 shap_values = explainer(X_train)

~Anaconda3libsite-packagesshapexplainers_explainer.py in __init__(self, model, masker, link, algorithm, output_names)
123 # if we get here then we don't know how to handle what was given to us
124 else:
--> 125 raise Exception("The passed model is not callable and is not any known model type: " + str(model))
126
127 # build the right subclass

Exception: The passed model is not callable and is not any known model type: Pipeline(memory=None,
steps=[('preprocessor',
ColumnTransformer(n_jobs=None, remainder='drop',
sparse_threshold=0.3,
transformer_weights=None,
transformers=[('num',
Pipeline(memory=None,
steps=[('imputer',
SimpleImputer(add_indicator=False,
copy=True,
fill_value=None,
missing_values=nan,
strategy='median',
verbose=0)),
('scaler',
StandardScaler(copy=True,
with_mean...
RandomForestRegressor(bootstrap=True, ccp_alpha=0.0,
criterion='mse', max_depth=6,
max_features='auto', max_leaf_nodes=None,
max_samples=None,
min_impurity_decrease=0.0,
min_impurity_split=None,
min_samples_leaf=1, min_samples_split=2,
min_weight_fraction_leaf=0.0,
n_estimators=100, n_jobs=None,
oob_score=False, random_state=0,
verbose=0, warm_start=False))],
verbose=False)

[Lamiane]

I think with model agnostic explainer you need to pass a function responsible for giving predictions not the model itself. Try:
explainer = shap.Explainer(model.predict, X_train)

At least it worked for me with KernelExplainer in ver 0.35

[Sajenthan]

Thanks @Lamiane I'm getting error in shap_values = explainer(X_train)

[Lamiane]

I don't know what your data is and how your code exactly works but I assume X_train contains sth that is not a number. A string maybe?

It seems that the problem is raised by the isfinite function from numpy. You can try to run it on each row of X_train to see which row causes the error.