Shap: No more PyTorch support

[GillesVandewiele]

PyTorch is no longer supported, the provided notebook will not execute successfully (https://github.com/slundberg/shap/blob/7391a95c4205dea651bf2ed839bf7734d2b83c8d/notebooks/deep_explainer/PyTorch%20Deep%20Explainer%20MNIST%20example.ipynb), due to this type checking:

https://github.com/slundberg/shap/blob/7391a95c4205dea651bf2ed839bf7734d2b83c8d/shap/explainers/gradient.py#L130

Btw, great library and tool!

[slundberg]

Thanks. The line you highlight is from the TF gradient explainer object, not the PyTorch one. The following lines should detect the framework: https://github.com/slundberg/shap/blob/7391a95c4205dea651bf2ed839bf7734d2b83c8d/shap/explainers/gradient.py#L55-L67

Perhaps they are not getting it right in your case/version of pytorch?

[GillesVandewiele]

Ok, thanks. I just cloned the repository and tried to execute the notebook and got an exception:

AssertionError                            Traceback (most recent call last)
<ipython-input-3-cfc22a02245d> in <module>()
      6 test_images = images[100:103]
      7 
----> 8 e = shap.DeepExplainer(model, background)
      9 shap_values = e.shap_values(test_images)

/usr/local/lib/python3.6/dist-packages/shap/explainers/deep.py in __init__(self, model, data, session, learning_phase_flags)
    102             self.model_output = model[1]
    103         else:
--> 104             assert False, str(type(model)) + " is not currently a supported model type!"
    105         assert type(self.model_output) != list, "The model output to be explained must be a single tensor!"
    106         assert len(self.model_output.shape) < 3, "The model output must be a vector or a single value!"

AssertionError: <class '__main__.Net'> is not currently a supported model type!

Also, quick question, would it be possible to calculate shapley values for categorical values that first go through embedding layers?

---

My version of Torch is 0.4.1

[slundberg]

Sorry for the slow reply. There is no deep.py file anymore in the latest version, so I wonder if the cloning worked or if you still have an old version somewhere? You can do shap.__version__

[austinmw]

Same issue for me -- I installed shap 0.25.0.
Although shap.__version__ returns AttributeError: module 'shap' has no attribute '__version__'

Using PyTorch 1.0:
e = shap.DeepExplainer(model, background) returns AssertionError: <class '__main__.Net'> is not currently a supported model type!

str(type(model)) returns "<class '__main__.Net'>". It looks like you're checking that it's either a Keras model or elif str(type(model)).endswith("tuple'>"):, which is not the case. Is this what PyTorch used to return in a previous version?

[slundberg]

@austinmw what line does that error come from? Keras or tuple is what we check for in the TF explainer so I wonder if the routing is not working at:
https://github.com/slundberg/shap/blob/1330bd979e0fb2e943c44e99231e579c9b4f0999/shap/explainers/deep/__init__.py#L63-L80

[andrewnaguib]

@GillesVandewiele Would you please change the issue's title? Because it's completely misleading.