Flair: NER task using Flair BertEmbeddings VS HuggingFace scripts
Hi everyone!
I am new to NLP and NER so I'm still trying to understand how exactly different architectures work.
My question is the following: the architecture implemented to do NER using Flair BertEmbeddings withing Flair SequenceTagger is the same implemented by HuggingFace Team in the pytorch/tf example scripts here?
In particular my doubt is related to the fact the Flair SequenceTagger is based on Bi-LSTM(-CRF) that I still see through layers when running, while HuggingFace scripts are based purely on the Transformer architecture.
I am running this tutorial in Google Colab.
I'd really appreciate any clarification. Thank you all in advance.
Best regards.
ChessMateK
All 3 comments
Maybe I have to ask to the master :-) @alanakbik
ChessMateK
on 8 Apr 2020
For the Huggingface scripts @stefan-it is the person to ask :)
Both implementations are very different: In Flair, our default sequence labeling architecture is BiLSTM-CRF with a feature-based approach (i.e. no fine-tuning of transformers) trained with many epochs of SGD and annealing. Huggingface is I believe doing a fine-tuning of transformers as in the BERT paper (few epochs, very small learning rate, Adam optimizer) which is very different.
We are just now adding this fine-tuning transformers approach for Flair as well - it's part of master branch and undergoing testing (see #1494), so it will be part of the next release. It should allow the community to directly compare both approaches.
alanakbik
on 8 Apr 2020
Thank you @alanakbik :)
ChessMateK
on 15 Apr 2020
Related issues
inyukwo1
路
3Comments
UrszulaCzerwinska
路
3Comments
shoarora
路
3Comments
mnishant2
路
3Comments
alanakbik
路
3Comments
Most helpful comment
For the Huggingface scripts @stefan-it is the person to ask :)
Both implementations are very different: In Flair, our default sequence labeling architecture is BiLSTM-CRF with a feature-based approach (i.e. no fine-tuning of transformers) trained with many epochs of SGD and annealing. Huggingface is I believe doing a fine-tuning of transformers as in the BERT paper (few epochs, very small learning rate, Adam optimizer) which is very different.
We are just now adding this fine-tuning transformers approach for Flair as well - it's part of master branch and undergoing testing (see #1494), so it will be part of the next release. It should allow the community to directly compare both approaches.