REALM: Retrieval-Augmented Language Model Pre-training - Super Knowledge Retriever

This paper introduces a way to use unsupervised learning to train a neural knowledge retriever that achieves state-of-the-art results in three popular Open-QA benchmarks, and it outperforms all previous methods by a significant margin (4-16