Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedback

• DOI: 10.48550/arxiv.2309.15762
• Publication Date: 2023-01-01
• Authors: Teresa Yeo; Oğuzhan Fatih Kar; Zahra Sodagar; Amir Zamir
• Keywords: Robustness

We propose a method for adapting neural networks to distribution shifts at test-time. In contrast training-time robustness mechanisms that attempt anticipate and counter the shift, we create closed-loop system make use of test-time feedback signal adapt network on fly. show this loop can be effectively implemented using learning-based function, which realizes an amortized optimizer network. This leads adaptation method, named Rapid Network Adaptation (RNA), is notably more flexible orders magnitude faster than baselines. Through broad set experiments various signals target tasks, study efficiency flexibility method. perform evaluations datasets (Taskonomy, Replica, ScanNet, Hypersim, COCO, ImageNet), tasks (depth, optical flow, semantic segmentation, classification), (Cross-datasets, 2D 3D Common Corruptions) with promising results. end discussion general formulations handling our observations from comparing similar approaches other domains.