$k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference

• DOI: 10.48550/arxiv.2303.13824
• Publication Date: 2023-01-01
• Authors: Benfeng Xu; Quan Wang; Zhendong Mao; Yajuan Lyu; Qiaoqiao She; Yongdong Zhang
• Keywords: Ranging

In-Context Learning (ICL), which formulates target tasks as prompt completion conditioned on in-context demonstrations, has become the prevailing utilization of LLMs. In this paper, we first disclose an actual predicament for typical usage that it can not scale up with training data due to context length restriction. Besides, existing works have shown ICL also suffers from various biases and requires delicate calibration treatment. To address both challenges, advocate a simple effective solution, $k$NN Prompting, queries LLM distributed representations, then predicts test instances by simply referring nearest neighbors. We conduct comprehensive experiments demonstrate its two-fold superiority: 1) Calibration-Free: Prompting does directly align output distribution task-specific label space, instead leverages such instances. It significantly outperforms state-of-the-art calibration-based methods under comparable few-shot scenario. 2) Beyond-Context: further effectively many are available, continually bringing substantial improvements. The scaling trend holds across 10 orders magnitude ranging 2 shots 1024 well different LLMs scales 0.8B 30B. successfully bridges into model scaling, brings new potentials gradient-free paradigm deployment. Code is publicly available.