Flair: Add support for Transformer-XL

Created on 19 Feb 2019  路  5Comments  路  Source: flairNLP/flair

Thanks to pytorch-pretrained-BERT, we can add a new embeddings class for Transformer-XL embeddings (idea mainly taken from #68).

picture stefan-it
🎉2

Most helpful comment

Training is also working on my end - I'm doing some example training runs. I'll merge into 0.4.1 branch in a bit once first numbers come in!

picture alanakbik on 19 Feb 2019
🎉2 👍2

All 5 comments

Awesome!

alanakbik picture alanakbik on 19 Feb 2019

I'm not sure if the PR should land into the upcoming 0.4.1 release 馃

Training is working :) Previous experiments I made resulted in a F-Score of 90 - 91 on CoNLL 2003 (English). I'm currently running a new experiment + I will share the results here soon!

stefan-it picture stefan-it on 19 Feb 2019

Hey wow that was fast :)

alanakbik picture alanakbik on 19 Feb 2019

Training is also working on my end - I'm doing some example training runs. I'll merge into 0.4.1 branch in a bit once first numbers come in!

alanakbik picture alanakbik on 19 Feb 2019
🎉2 👍2

Transformer-XL embeddings can be used in 0.4.1 now, so I'm closing here :)

stefan-it picture stefan-it on 22 Feb 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

shoarora picture shoarora  路  3Comments
prematurelyoptimized picture prematurelyoptimized  路  3Comments
jannenev picture jannenev  路  3Comments
inyukwo1 picture inyukwo1  路  3Comments
jewl123 picture jewl123  路  3Comments