Transformers: How to use fine-tuned BERT to fill <mask>

[Odimmsun]

❓ Questions & Help

Details

I have fine-tuned a BERT model by classification task, use transformers.BertForSequenceClassification. Now, i want use this model to fill mask when i give a input like: 'my dog

Can this be implemented?
I would really appreciate if someone can teach me how to implement.

[patil-suraj]

Hi @Odimmsun , BertForSequenceClassification is used for classification task and not for mak-filling. You can use the pre-trained BERT model as it is for make-filling. There is pipeline for this task which you can find here https://huggingface.co/transformers/usage.html#masked-language-modeling

Basically what you'll need to do is this

from transformers import pipeline

nlp = pipeline("fill-mask")
print(nlp(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks."))

you can also pass your own model to the pipeline using the model parameter.

[Odimmsun]

Hi @Odimmsun , BertForSequenceClassification is used for classification task and not for mak-filling. You can use the pre-trained BERT model as it is for make-filling. There is pipeline for this task which you can find here https://huggingface.co/transformers/usage.html#masked-language-modeling

Basically what you'll need to do is this

from transformers import pipeline

nlp = pipeline("fill-mask")
print(nlp(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks."))

you can also pass your own model to the pipeline using the model parameter.

Hi @patil-suraj ,thank you for your reply. Now, if i use "nlp = pipeline("fill-mask")", the model i used is not fine-tuned, but i want to use a fine-tuned model to fill mask. How should i do to implement this?

[patil-suraj]

The pre-trained BERT model does mask-filling out of the box, but if you want to use your own fine-tuned model then just pass the model(path or url) to the model parameter .

nlp = pipeline("fill-mask", model="your_model_path")

[Odimmsun]

`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
1 my_model.to('cpu')
2 nlp = pipeline(task='fill-mask', model=my_model, tokenizer=tokenizer)
----> 3 nlp('我是你爸爸')

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in __call__(self, args, *kwargs)
804 values, predictions = topk.values.numpy(), topk.indices.numpy()
805 else:
--> 806 masked_index = (input_ids == self.tokenizer.mask_token_id).nonzero().item()
807 logits = outputs[i, masked_index, :]
808 probs = logits.softmax(dim=0)

ValueError: only one element tensors can be converted to Python scalars
`
hi, my model is fine-tuned by BertForSequenceClassification. Then i use it to fill mask, the error raised as above. Now， i am confused.

[patil-suraj]

BertForSequenceClassification is meant for classification, it won't work for mask-filling task. If you want to fine-tune BERT for masked lm task then you should use BertForMaskedLM

[patil-suraj]

Check this example for how to fine-tune bert for masked LM

https://github.com/huggingface/transformers/tree/master/examples/language-modeling#robertabertdistilbert-and-masked-language-modeling

[orena1]

Thank @patil-suraj, do you know how to use BART with in-filling scheme (where spans of text are replaced with a single mask token)? I have not seen this pipline

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[FrancisYu2020]

`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
1 my_model.to('cpu')
2 nlp = pipeline(task='fill-mask', model=my_model, tokenizer=tokenizer)
----> 3 nlp('我是你爸爸')

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in call(self, args, *kwargs)
804 values, predictions = topk.values.numpy(), topk.indices.numpy()
805 else:
--> 806 masked_index = (input_ids == self.tokenizer.mask_token_id).nonzero().item()
807 logits = outputs[i, masked_index, :]
808 probs = logits.softmax(dim=0)

ValueError: only one element tensors can be converted to Python scalars
`
hi, my model is fine-tuned by BertForSequenceClassification. Then i use it to fill mask, the error raised as above. Now， i am confused.

老哥稳！