Transformers: REALM

[aced125]

🌟 New model addition

REALM is from some of the authors of BERT (I like to think of it as the next BERT :) ) that have found a way to incorporate world knowledge (from Wikipedia) into the model.

They do this by having the concept of a retriever module that retrieves information from wikipedia articles.

Open source status

Code not released at the moment but will probably be released by Google soon I'd imagine.

https://arxiv.org/abs/2002.08909

[AlecS12]

Code is released for the preceding paper: Latent Retrieval for Weakly Supervised Open Domain Question Answering" https://www.aclweb.org/anthology/P19-1612/
https://github.com/google-research/language/tree/master/language/orqa

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[antoniolanza1996]

I am also interested on it. Any news? Thanks

[alexvaca0]

I'm also interested on this, are you planning to include this model in the library??

[antoniolanza1996]

Finally the REALM code has been released here https://github.com/google-research/language/tree/master/language/realm.
I think that this issue should be re-opened. @aced125 are you able to re-open it?

[Utomo88]

I hope somebody can reopen this and make it happen

[krrishdholakia]

Any updates on this ?