Captum: Captum for Bert Sentence Classification

[diogocortiz]

Hi there,

I tried to apply Captum Tutorial for Q&A to Bert Sentence Classification task, but I am facing difficulties to adapt baselines / references part of the code for Classification and the new HugginFace Tokenizer.

Just want to check if someone is working in the same topic, so we can share experiences.

[NarineK]

Hi @diogocortiz,

It shouldn't be problematic to adjust the baselines. Someone in our previous threads used it for a Classification task: https://github.com/pytorch/captum/issues/150#issuecomment-566249109 (btw you don't need to override forward function as mentioned in the issue. We've fixed that.)

What is the exact issue that you're facing with BERT ?

[diogocortiz]

Never mind. I fixed the problem. =D

I've just started working with Captum, so I have a doubt.
What do you mean by Start and End Position? Is it related to prior and after Transformers layers? Or is it related when I have two sentences (such as Q&A). I accessed the github code, but I couldn't figure out why Addition_foward_args is 0 for the start and 1 for the end. Example:

attributions_start, delta_start = lig.attribute(inputs=input_ids, baselines=ref_input_ids,
                                additional_forward_args=(token_type_ids, position_ids, attention_mask, 0),
                                return_convergence_delta=True)

attributions_end, delta_end = lig.attribute(inputs=input_ids, baselines=ref_input_ids,
                                additional_forward_args=(token_type_ids, position_ids, attention_mask, 1),
                                return_convergence_delta=True)

If I run the code above, I get the same output for importance (the same viz also).
But if I omit the additional_forward_args for one of the sentences, I got two differents outputs (and two differents viz also)

Thank you so much!

[NarineK]

Glad that you found the issue, @diogocortiz !

With start and end positions I meant the start and end positions of the predicted answer in the text.
The model outputs 2 values one is the start position of the answer in the text and the other one is the end position of the answer.
In my example, I attribute to the start position using my custom forward function squad_pos_forward_func and the index 0 and to the end position with index 1. I'm able to do that using the custom function that I have.
Basically, pred[0] accesses the prediction for the start position and pred[1] to the end position.
This way I can find out which tokens specifically contributed to the prediction of the start and which tokens to the end positions of the answer in the text.

But maybe you have a different version of Bert model and that's why there might be inconsistencies ?
Btw, with this PR: https://github.com/pytorch/captum/pull/247 I'm adding more stuff to it and did some clean up. This PR also uses a bert-base-uncased model as a pre-trained model.

[NarineK]

@diogocortiz! Were you able to find the problem ? If so, can we close this issue ?

[diogocortiz]

Hi @NarineK! Yes, I found the problem in my code. I fixed it in my code and now I am able to use captum. Thank you so much for your attention.

Just one question to make sure I'm not missing anything. I just started using Captum.
In my case, in Sentece Classification model, is the attribution for each token related to the importance for that prediction, right? Higher numbers (green) is more important tokens while lower numbers (red) is less important regardless the result of prediction (true or positive, for example), right?

I am asking you because I took a look in the Text Classifcation tutorial in Captum.ai website and I noticed in that case that it sees higher numbers (green) were more important for positive sentiment classifcation and lower numbers (red) were more important for negative sentiment. So I got a little confused.
By the way, how can I interpret the attribution score? Is it a mean of all attributions?

Again, thank you so much for your attention and help!!!

BR,
Diogo

[NarineK]

Hi @diogocortiz ! Glad that you're able to find the issue!

In the Sentence classification example that we have on captum.ai website for IMDB dataset, our model has a single output. That output is the prediction probability (p) of being a positive sentiment. A negative sentiment would be (1 - p).
In our case we attribute positive sentiment probability (p) to the inputs of our model and in case something is predicted with high probability as positive sentiment we see many tokens that contribute positively to the positive sentiment.

In case when p is very low, there are no words contributing to the positive sentiment and when we attribute to the positive sentiment prob (p) we find words that pull away (influence negatively) to the positive sentiment. Those tokens are obviously the ones that pull towards the negative sentiment with higher (1-p) probability.

Now, let me show a trick. Change the cell content of 12 to:

def forward_with_sigmoid_experiment(input):
    return 1 - torch.sigmoid(model(input))
lig = LayerIntegratedGradients(forward_with_sigmoid_experiment, model.embedding)

and run the example. You'll observe the opposite effect because I'm now attributing to the negative sentiment (1-p):
This is what you'll see:

Does this make sense ?

[jaguarpaw249]

Hi @NarineK @diogocortiz
I'm working on a multi-class classification problem. The custom foward function is similar to the forward_with_sigmoid_experiment shared by you. It will output the results between 0-1 for a particular class. The function is as follows:

def custom_forward_func(inputs, token_type_ids=None, position_ids=None, attention_mask=None, position=0):
    outputs = predict(inputs, token_type_ids=token_type_ids, position_ids=position_ids, attention_mask=attention_mask)
    preds = outputs[0].detach().cpu().numpy()
    return torch.tensor([softmax(preds)[0][position]]) # Output is between 0 - 1; position is class index

lig = LayerIntegratedGradients(custom_forward_func, model.bert.embeddings)

attributions, delta = lig.attribute(inputs=input_ids, baselines=ref_input_ids, additional_forward_args=(token_type_ids, position_ids, attention_mask, 1), return_convergence_delta=True)

I'm getting the following error:

[diogocortiz]

Yes, it does make sense now. Thank you for your explanation.
I am getting more familiar with Captum. =D

In my case, I have only two classes Hate Speech (1) and Not Hate Speech (0). In that context, if my model predicts a sentence as Hate Speech (1), then I can understand the green tokens (high attributes) as the tokens that most contributed for that prediction (1) and the red tokens (negative tokens) as the one that are pulling it towards another prediction (0), right?

And in all cases the attribution is calculated based on the prediction and not based on the ground truth (label). For example: if my model predicts a sentence as hate speech (1), but In fact the label was (0) - not hate, then the attributions are telling me the tokens that are contributing to the sentence being predicted as (1) hate and not (0) not hate". Is that correct?

thank you again,

Diogo

[NarineK]

Hi @diogocortiz ! You're welcome :)
That's right! In a general case, red means that those tokens are pulling away from the Hate Speech (1) and most probably pulling towards the opposite class however I think that red might not always mean that it will always attribute to the other class. I think that's the assumption that we make here. We assume that the classifier is able to identify that a token is negatively correlated with the Hate Speech(1) class so it must know something about that token, namely, that it is strongly pulling towards the opposite class (because there are no other options) and this is much easier to imagine for 2 class problem.

Actually you can experiment with attributing p and (1-p) to the inputs and see if you are getting the same magnitude of attributions in opposite signs. :)

Regarding the second question: yes, that's right!

[NarineK]

@jaguarpaw249 , this looks like a different problem. Perhaps you can create a separate issue for it ? Please do not get the numpy value here, use the tensor:

preds = outputs[0].detach().cpu().numpy()

[jaguarpaw249]

@NarineK
Thanks for the reply.
I have solved the issue :)

[diogocortiz]

Ni @NarineK, thank you much for your amazing support.
I am working a research regarding hatespeech in Portuguese and Captum will help us understanding the model, prediction and our data itself. I will share with you our outcomes later on.
I will close this case now. Thank you again.

[NarineK]

Thank you too, @diogocortiz ! I'd love to hear about your findings ! Keep us posted!

[efatmae]

@diogocortiz, I'm working on a similar problem like yours but detecting cyberbullying as a binary classification problem using BERT. I've not managed to use Captum with sentence classification successfully yet. Can you share the part where you sue Captum here??

Thanks