Allennlp: Add a TokenEmbedder that uses OpenAI's language model

[matt-gardner]

This would let us directly compare our ELMo models vs. their model, and would let others use whichever gives better performance. Code is here: https://github.com/openai/finetune-transformer-lm.

This is probably actually pretty challenging, as it would require duplicating their transformer architecture exactly, and loading the weights in correctly from their pre-trained files.

[leonweber]

You probably already know this, but there is already a reimplementation of the model in pytorch: https://github.com/huggingface/pytorch-openai-transformer-lm

[matt-gardner]

This is done.

[qiuwei]

@matt-gardner Could you share a config file which uses openai-transformer-embedder in the training-configs?

Thanks in advance.