Captum: Constructing reference/baseline for LSTM - Layer Integrated Gradients

[devpramod]

`# compute attributions and approximation delta using layer integrated gradients

attributions_ig, delta = lig.attribute((input_indices,h), (reference_indices,h), \
n_steps=500, return_convergence_delta=True)
`
When I run the above code, I get the following error.
AssertionError: baseline input argument must be either a torch.Tensor or a number however detected

But the documentation says that a tuple can be passed as baseline argument.

Am I doing this right?

[bilalsal]

Hi @devpramod ,

Does the tuple (reference_indices,h) represent your attribution target?

In that case, maybe specify it as a keyword argument.

attributions_ig, delta = lig.attribute((input_indices,h), target = (reference_indices,h),
n_steps=500, return_convergence_delta=True)

Treated as positional, argument, lig.attribute() expects this parameter to be the optional baselines.
Find out more at https://captum.ai/api/layer.html#layer-integrated-gradients

Hope this helps

[devpramod]

No, it represents my baseline/reference.
I'm calculating it the following way:

 PAD_IND = TEXT.vocab.stoi['pad']
 token_reference = TokenReferenceBase(reference_token_idx=PAD_IND)
 vis_data_records_ig = []

def interpret_sentence(model, sentence, min_len = 7, label = 0):
   text = [tok.text for tok in nlp.tokenizer(sentence)]
   if len(text) < min_len:
       text += ['pad'] * (min_len - len(text))
   indexed = [TEXT.vocab.stoi[t] for t in text]

   model.zero_grad()

   input_indices = torch.tensor(indexed, device=device)
   input_indices = input_indices.unsqueeze(0)

   # input_indices dim: [sequence_length]
   seq_length = min_len

   # predict
   pred = forward_with_sigmoid(input_indices).item()
   pred_ind = round(pred)

   # generate reference indices for each sample
   [1] reference_indices = token_reference.generate_reference(seq_length, device=device).unsqueeze(0)

   # compute attributions and approximation delta using layer integrated gradients
   [2] attributions_ig, delta = lig.attribute(input_indices, reference_indices, \
                                       n_steps=500, return_convergence_delta=True)

   print('pred: ', Label.vocab.itos[pred_ind], '(', '%.2f'%pred, ')', ', delta: ', abs(delta))

   add_attributions_to_visualizer(attributions_ig, text, pred, pred_ind, label, delta, vis_data_records_ig)

I've marked the important lines in the function using [1] and [2]. Except that I need to incorporate the hidden state into the baseline and I'm not sure how to.
I'm following https://captum.ai/tutorials/IMDB_TorchText_Interpret

[bilalsal]

Hi @devpramod ,

thanks for providing the helpful code and apologies for misreading the title.

Yes, the baseline parameter can be a tuple, however, _each_ tuple element should be a Tensor, int, or a float.

Does this apply to the hidden state h in your (reference_indices,h) tuple?
From the error message you get I suspect that h is of class 'tuple', because reference_indices is guaranteed to be a torch Tensor.
If this is indeed the case, all you need is to think of a useful way to blend h in reference_indices so that we can get a tensor of that serves as a meaningful baseline.
Maybe we can think of some ways if you could elaborate on your motivation to incorporate the hidden state into the baseline in the first place.
Also, is it possible that Neuron Integrated Gradient is a better fit for your use case?

[miguelmartin75]

A tuple can be passed as a baseline (the tuple represents multiple inputs to the model), but this tuple is should effectively be a Tuple[Union[PythonScalar, Tensor], ...]. Based off your error, it seems that your h is actually a tuple (it's not clear from your comments but perhaps you're doing out, h = model.lstm(x) rather than out, (h, c) = model.lstm(x)).

Note that the inputs to LayerIG will be fed to the model directly. So if you want to pass in the hidden state you'd have to handle that in your model's forward function.

However, with LayerIntegratedGradients, if given the LSTM layer, it should patch the last hidden state for you (i.e. the output of the LSTM Layer; it will also patch the other outputs as well - specifically the last cell state and output of the lstm itself).

I am assuming the last hidden state is what you want to patch, since this is the most common use-case. Unless you need to set the hidden state(s) to something specific, you should be able to just use LayerIntegratedGradients in the same way the tutorial shows.

If the case is you want to set the hidden state(s) to something else (i.e. a hidden state that does not correspond to the reference indices), e.g. a random vector, then I think you have a couple of options:

1. Feed the hidden state(s) to a dummy module (e.g. the identity) and pass this dummy module to LayerIntegratedGradients as the layer. Additionally you'd have to have a flag in this dummy module to output your desired baseline for the hidden state(s).
2. Do what LayerIntegratedGradients does manually

I think (1) is likely your best/easiest option in this case.

[devpramod]

Thank you very much for your reply. I'll try these out as soon as possible and report back here to close the issue.