Transformers: Questions Answering Example

[schipiga]

Hello folks! Can you provide simple example how to use pytorch bert with pretrained model for questions answering?

[rodgzilla]

Hi!

You can check this file that implements question answering on the SQuAD dataset: https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_squad.py

[schipiga]

Thank you very much!

[adilmukhtar82]

How can we use pre trained BertForQuestionAnswering model? I have looked into BertForNextSentencePrediction and output of model makes sense given the input vector, but unable to find any good example on BertForQuestionAnswering.

[LysandreJik]

Have you tried looking at the official documentation that provides a simple example for each model?

[Arjunsankarlal]

Hi @LysandreJik, the official example was not clear to me. I understood the part of encoding. But I am looking for something like, I will give a question and a paragraph which would contain the answer, and I need the model to predict the answer span. But in the example they have done it with a single sentence, which is quite confusing!

[Arjunsankarlal]

Hey @LysandreJik, Sorry my bad, didn't look at run_squad.py, it has been changed a lot since I saw it first during which BERT was only released! It is so good to see everything being integrated at a single place! Thanks for the great work you guys! ❤️

[LysandreJik]

@Arjunsankarlal Glad you could get what you were looking for!

[adilmukhtar82]

@Arjunsankarlal @LysandreJik can you guys help me with the example. I got an error when I ran the example given in the documentation when encoding the sequence, that tokernizer doesn't have attribute "encode". So I updated the code as follows:

`from pytorch_pretrained_bert import BertTokenizer, BertForQuestionAnswering
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForQuestionAnswering.from_pretrained('bert-base-uncased')

tokenized_text = tokenizer.tokenize("Hello, my dog is cute")

indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)

input_ids = torch.tensor([indexed_tokens]) # Batch size 1
start_positions = torch.tensor([1])
end_positions = torch.tensor([3])
outputs = model(input_ids, start_positions=start_positions, end_positions=end_positions)
print(outputs)`

this is the output
tensor(1.7739, grad_fn=)

I believe it's a loss but I don't understand the example as in how does it answer the question. Also there isn't any start and end span. Can you please explain the example. Much appreciated.

[LysandreJik]

Hi @adilmukhtar82 , could you give a look at the run_squad.py example, it shows how to use several models to do question answering.

You should probably update your repository version to pytorch-transformers too, most of the examples on our documentation won't work with pytorch_pretrained_bert.

[adilmukhtar82]

@LysandreJik Thanks I have updated the repository and example is working fine. I am confused about the example mentioned in documentation ("hello, my dog is cute") as to how does it do with single sentence and not paragraph along with it.

[ramsrigouthamg]

A bit late but here you go -

from transformers import DistilBertTokenizer, DistilBertForQuestionAnswering
import torch

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased',return_token_type_ids = True)
model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased-distilled-squad')

context = "The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world."
question = "What was President Donald Trump's prediction?"
encoding = tokenizer.encode_plus(question, context)

input_ids, attention_mask = encoding["input_ids"], encoding["attention_mask"]
start_scores, end_scores = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))

ans_tokens = input_ids[torch.argmax(start_scores) : torch.argmax(end_scores)+1]
answer_tokens = tokenizer.convert_ids_to_tokens(ans_tokens , skip_special_tokens=True)

all_tokens = tokenizer.convert_ids_to_tokens(input_ids)

print ("\nAnswer Tokens: ")
print (answer_tokens)

answer_tokens_to_string = tokenizer.convert_tokens_to_string(answer_tokens)

print ("\nFinal Answer : ")
print (answer_tokens_to_string)

Output is :
Answer Tokens:
['some', 'states', 'would', 're', '##open', 'this', 'month']

Final Answer :
some states would reopen this month

[mariusjohan]

@ramsrigouthamg Hey, could you maybe also provide a tensorflow example?

[LysandreJik]

Thanks @ramsrigouthamg !

@mariusjohan there are PyTorch and TensorFlow examples in the usage section of the documentation.

[kmair]

@LysandreJik The link is now updated to https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py