Matching-oriented Embedding Quantization For Ad-hoc Retrieval

• DOI: 10.18653/v1/2021.emnlp-main.640
• Publication Date: 2021-12-16T22:56:42Z
• Authors: Shitao Xiao; Zheng Liu; Yingxia Shao; Defu Lian; Xing Xie
• Keywords: 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

Product quantization (PQ) is a widely used technique for ad-hoc retrieval. Recent studies propose supervised PQ, where the embedding and models can be jointly trained with learning. However, there lack of appropriate formulation joint training objective; thus, improvements over previous non-supervised baselines are limited in reality. In this work, we Matching-oriented Quantization (MoPQ), novel objective Multinoulli Contrastive Loss (MCL) formulated. With minimization MCL, able to maximize matching probability query ground-truth key, which contributes optimal retrieval accuracy. Given that exact computation MCL intractable due demand vast contrastive samples, further Differentiable Cross-device Sampling (DCS), significantly augments samples precise approximation MCL. We conduct extensive experimental on four real-world datasets, whose results verify effectiveness MoPQ. The code available at https://github.com/microsoft/MoPQ.