Fairseq: Using sentencepiece tokenizer in RoBERTa pretraining regiment

Created on 3 Sep 2019  路  2Comments  路  Source: pytorch/fairseq

Would it be possible to use sentencepiece tokenizer in preprocessing the data?

picture nitinnairk

Most helpful comment

Is there an example to use sentencepiece tokenizer in preprocessing the data?

picture simonefrancia on 6 Dec 2019
👍2

All 2 comments

Hey @nitinnairk it's definitely possible if you are pretraining from scratch.
The released pretrained model used GPT2 bpe dictionary so unfortunately you can't use sentencepiece tokenizer with the released model.

ngoyal2707 picture ngoyal2707 on 2 Oct 2019
👍1

Is there an example to use sentencepiece tokenizer in preprocessing the data?

simonefrancia picture simonefrancia on 6 Dec 2019
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

prihoda picture prihoda  路  3Comments
mali-nuist picture mali-nuist  路  3Comments
gaopengcuhk picture gaopengcuhk  路  3Comments
mjpost picture mjpost  路  3Comments
galphag picture galphag  路  3Comments