Pre-trained BERT Model Retrieval: Inference-Based No-Learning Approach using k-Nearest Neighbour Algorithm

• DOI: 10.1587/transinf.2024dat0003
• Publication Date: 2025-01-06T22:11:29Z
• Authors: Huu-Long PHAM; Ryota MIBAYASHI; Takehiro YAMAMOTO; Makoto P. KATO; Yusuke YAMAMOTO; Yoshiyuki SHOJI; Hiroaki OHSHIMA

In this study, we propose a method to efficiently retrieve BERT pre-trained models that achieve good performance on specific document classification task. natural language processing problems, the common practice involves fine-tuning existing rather than building new ones from ground up due extensive time and computational resources required. The challenge, however, lies in identifying most suitable model large number of available models. To address problem, our proposed utilizes k-nearest neighbor algorithm appropriate without necessity for fine-tuning. We conducted experiments by constructing benchmark dataset with 28 tasks 20