Captum: DeepLiftShap example doesnt work on Imagenet data

[giangnguyen2412]

My code is:

import numpy as np

import torch

from captum.attr import (
    GradientShap,
    DeepLift,
    DeepLiftShap,
    IntegratedGradients,
    LayerConductance,
    NeuronConductance,
    NoiseTunnel,
)

model = models.resnet50(pretrained=True).eval()
print(test_images.shape)
print(background.shape)

dl = DeepLiftShap(model)
attributions, delta = dl.attribute(inputs=test_images, baselines=background)
print('DeepLiftSHAP Attributions:', attributions)
print('Convergence Delta:', delta)

🐛 Log

torch.Size([10, 3, 224, 224])
torch.Size([30, 3, 224, 224])

---

TypeError Traceback (most recent call last)
in
18
19 dl = DeepLiftShap(model)
---> 20 attributions, delta = dl.attribute(inputs=test_images, baselines=background)
21 print('DeepLiftSHAP Attributions:', attributions)
22 print('Convergence Delta:', delta)

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_core/deep_lift.py in attribute(self, inputs, baselines, target, additional_forward_args, return_convergence_delta, custom_attribution_func)
764 Literal[True, False], return_convergence_delta
765 ),
--> 766 custom_attribution_func=custom_attribution_func,
767 )
768 if return_convergence_delta:

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_core/deep_lift.py in attribute(self, inputs, baselines, target, additional_forward_args, return_convergence_delta, custom_attribution_func)
320 self.model, (inputs, baselines), expanded_target, input_base_additional_args
321 )
--> 322 gradients = self.gradient_func(wrapped_forward_func, inputs,)
323 if custom_attribution_func is None:
324 attributions = tuple(

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_utils/gradient.py in compute_gradients(forward_fn, inputs, target_ind, additional_forward_args)
94 with torch.autograd.set_grad_enabled(True):
95 # runs forward pass
---> 96 outputs = _run_forward(forward_fn, inputs, target_ind, additional_forward_args)
97 assert outputs[0].numel() == 1, (
98 "Target not provided when necessary, cannot"

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_utils/common.py in _run_forward(forward_func, inputs, target, additional_forward_args)
490 forward_func_args = signature(forward_func).parameters
491 if len(forward_func_args) == 0:
--> 492 output = forward_func()
493 return output if target is None else _select_targets(output, target)
494

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_core/deep_lift.py in forward_fn()
354 ) -> Callable:
355 def forward_fn():
--> 356 return _run_forward(forward_func, inputs, target, additional_forward_args)
357
358 if hasattr(forward_func, "device_ids"):

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/captum/attr/_utils/common.py in _run_forward(forward_func, inputs, target, additional_forward_args)
501 (inputs, *additional_forward_args)
502 if additional_forward_args is not None
--> 503 else inputs
504 )
505 return _select_targets(output, target)

~/miniconda3/envs/advanced_ml/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, input, *kwargs)
491 result = self._slow_forward(input, *kwargs)
492 else:
--> 493 result = self.forward(input, *kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)

TypeError: forward() takes 2 positional arguments but 3 were given

Can some one help me out?

[vivekmig]

Hi @giangnguyen2412 , from the error, it seems like there could be an issue with the model forward. Is the model just imported from the torchvision pretrained ResNet50 or is this a custom model with any modifications?
Can you also confirm if running model(test_images) runs without any errors?

[giangnguyen2412]

This is a torchvision pretrained ResNet50 and yep, i can run model(test_images) without any errors. Can you look at my code and give me advice. Thanks.

[vivekmig]

Hi @giangnguyen2412 , from a closer look at the stack trace, I think it's possible you might be using torch version 1.1, which is not supported by Captum. Can you double check the version of PyTorch that you are using with print(torch.__version__)?

If it is prior to version 1.2, can you upgrade to a newer version of PyTorch and try again?

[giangnguyen2412]

I upgraded to torch 1.5.1 and the error has gone. But I found another error here.

Note that with the same code, I can run vgg16, but resnet34 and 50 failed. Could you give me some hints?

[vivekmig]

Hi @giangnguyen2412 , the issue here is that the default torchvision ResNet model reuses a single ReLU modules multiple times, which is not supported by the DeepLift implementation. This issue describes an approach to fix the issue #378 .

[giangnguyen2412]

Hi @vivekmig , thanks for your help. I have looked at that, but does it mean I need to retrain the model?

[vivekmig]

Hi @giangnguyen2412, no problem! You shouldn't need to retrain the model if you just copy the source from here and make the change described in the issue above. After importing the modified ResNet, you should still be able to pass pretrained=True, and obtain the pretrained model appropriately.

[giangnguyen2412]

Hi @vivekmig , thanks for your help. I want to add some details into your advice (for those who are new). First, copy the definitions from here and execute them as your source. You should import two funcs load_state_dict_from_url and _get_torch_home from torch.hub. Then define your model as model=resnet50(pretrained=True).eval(). One more thing is that you are not only required to modifed the BasicBlock class but also Bottleneck class to make it work!

[giangnguyen2412]

@vivekmig , as I observed, the maps from both DeepLiftShap and GradientShap are not clear. Am I correctly doing if my result is sth like this. (as I dont have any reference, I surfed the Internet not found any results about DeepLiftShap with captum).

The used script is:

import numpy as np

import torch

from captum.attr import *

from captum.attr import (
    GradientShap,
    DeepLift,
    DeepLiftShap,
    IntegratedGradients,
    LayerConductance,
    NeuronConductance,
    NoiseTunnel,
)

model_m = resnet50(pretrained=True).eval().to(device)

dl = DeepLiftShap(model_m)

attributions = dl.attribute(inputs=test_images, baselines=background, target=3)

att_ndarray = attributions[0].detach().cpu().numpy()
att_ndarray = np.swapaxes(att_ndarray, 0, -1)
att_ndarray = np.swapaxes(att_ndarray, 0, 1)

input_ndarray = test_images[0].detach().cpu().numpy()
input_ndarray = np.swapaxes(input_ndarray, 0, -1)
input_ndarray = np.swapaxes(input_ndarray, 0, 1)


# print('DeepLiftSHAP Attributions:', attributions)
_ = visualization.visualize_image_attr_multiple(att_ndarray, input_ndarray, 
                                                ["original_image", "heat_map", "blended_heat_map", "masked_image", "alpha_scaling"],
                                               ["positive", "positive", "positive", "positive", "positive"])

[vivekmig]

Hi @giangnguyen2412 , your usage looks generally correct, I don't see any obvious issues. Just double check that the target=3parameter corresponds to the output you want to interpret for each input. This is often chosen to be the true class index, but can be used for other classes to see pixel importance for an alternate / incorrect decision, this FAQ answer provides more context.

When visualizing, It may also be worthwhile to look at both positive / negative or absolute value of attributions in addition to only positive signed results.

For more examples of using other methods, you can take a look at this tutorial.

[giangnguyen2412]

In addition to the comment of @vivekmig , you can try to increase the number of background samples to improve the quality of heatmaps