Multi-Stage Document Ranking with BERT

• DOI: 10.48550/arxiv.1910.14424
• Publication Date: 2019-01-01
• Authors: Rodrigo Nogueira; Wei Yang; Kyunghyun Cho; Jimmy Lin
• Keywords: Pointwise

The advent of deep neural networks pre-trained via language modeling tasks has spurred a number successful applications in natural processing. This work explores one such popular model, BERT, the context document ranking. We propose two variants, called monoBERT and duoBERT, that formulate ranking problem as pointwise pairwise classification, respectively. These models are arranged multi-stage architecture to form an end-to-end search system. One major advantage this design is ability trade off quality against latency by controlling admission candidates into each pipeline stage, doing so, we able find operating points offer good balance between these competing metrics. On large-scale datasets, MS MARCO TREC CAR, experiments show our model produces results either at or comparable state art. Ablation studies contributions component characterize latency/quality tradeoff space.