Retrain or not retrain: Conformal test martingales for change-point detection

• DOI: 10.48550/arxiv.2102.10439
• Publication Date: 2021-01-01
• Authors: Vladimir Vovk; Ivan Petej; Ilia Nouretdinov; Ernst Ahlberg; Lars Carlsson; Alex Gammerman
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; 68Q32 (Primary) 62G10, 60G42, 68T05 (Secondary); Statistics - Machine Learning; 0202 electrical engineering, electronic engineering, information engineering; Machine Learning (stat.ML); 02 engineering and technology; Machine Learning (cs.LG)

We argue for supplementing the process of training a prediction algorithm by setting up scheme detecting moment when distribution data changes and needs to be retrained. Our proposed schemes are based on exchangeability martingales, i.e., processes that martingales under any exchangeable data. method, conformal prediction, is general can applied top modern algorithm. Its validity guaranteed, in this paper we make first steps exploring its efficiency.