Allennlp: How to set the configuration file for using bert

Created on 14 Jan 2019  路  2Comments  路  Source: allenai/allennlp

Is there an example file for using bert with the configuration file?

For example, in a sequence tagging task, how to modify the configuration file in
(allennlp/tutorials/getting_started/walk_through_allennlp/simple_tagger.json) to change the lstm encoder with the embedding to the bert pretrained model?

Could you please show me an example?

  1. the sentence is original without split to word piece.
    "Lauren/np said/vbd ,/, / Why/wrb can't/md* I/ppss call/vb my/pp$ home/nr ?/. ?/.
    or
  2. the sentence is tokenized with word piece
    "[CLS]/[CLS] lauren/np said/vbd ,/, /`` ##/`` why/wrb can/md* ##'/md* ##t/md* i/ppss call/vb my/pp$ home/nr ?/. ?/. [SEP]/[SEP]"

Most helpful comment

here's the config I used to train an NER model using BERT embeddings, hope that's helpful:

https://gist.github.com/joelgrus/7cdb8fb2d81483a8d9ca121d9c617514

All 2 comments

here's the config I used to train an NER model using BERT embeddings, hope that's helpful:

https://gist.github.com/joelgrus/7cdb8fb2d81483a8d9ca121d9c617514

here's the config I used to train an NER model using BERT embeddings, hope that's helpful:

https://gist.github.com/joelgrus/7cdb8fb2d81483a8d9ca121d9c617514

Thanks very much!
It is really helpful!

Was this page helpful?
0 / 5 - 0 ratings