Allennlp: Update BERT-specific embedders to new pretrained transformer indexers and embedders

[schmmd]

• Drop dependencies on pytorch-pretrained-bert and pytorch-transformers (depend on transformers instead).
• SOTA models fine-tune transformers so we should make this easy. Specifically we should review transformers tutorials, verify that those tasks are easily accomplishable in AllenNLP, and write equivalent tutorials for AllenNLP.

[knighter]

That would be really useful, looking forward to see this feature!

[amolk]

@schmmd, are you close to being done with this? If not, I will have to roll up my sleeves to put something together for a project.

[matt-gardner]

I'm not actually sure why @schmmd put this as "Under Development". Maybe @brendan-ai2 has this on his radar to work on this quarter? I don't think anyone is actively working on this right now, and contributions are welcome.

There are a number of separate issues bundled in this, though, and some of them are probably tricky to wrap your head around if you're not really familiar with the library. We'd love to have contributions, just run whatever it is you're thinking by us before you start working on something, so that we can be sure you're on the right track.

[schmmd]

I should not have tagged this as Under Development. We plan to have this complete before the 1.0 launch, but we don't know when that will be exactly yet and nobody is currently scheduled to work on this. Contributions would be very welcome--but as Matt said it's best to discuss your approach with us first so there's no wasted effort.

[maksym-del]

This two belong here: https://github.com/allenai/allennlp/issues/3448, https://github.com/allenai/allennlp/issues/3450,

[matt-gardner]

We need to be sure that this also handles pooling for classification tasks.

[matt-gardner]

This was mostly finished in #3634. We still need to remove the old classes, and be sure that things work with a pooler, but otherwise this is done.