Benchmarking uncertainty quantification for protein engineering

• DOI: 10.1371/journal.pcbi.1012639
• Publication Date: 2025-01-07T18:45:57Z
• Authors: Kevin P. Greenman; Ava P. Amini; Kevin K. Yang
• Keywords: Benchmark (surveying); Uncertainty Quantification; Bayesian Optimization; Benchmarking

Machine learning sequence-function models for proteins could enable significant advances in protein engineering, especially when paired with state-of-the-art methods to select new sequences property optimization and/or model improvement. Such (Bayesian and active learning) require calibrated estimations of uncertainty. While studies have benchmarked a variety deep uncertainty quantification (UQ) on standard molecular machine-learning datasets, it is not clear if these results extend datasets. In this work, we implemented panel UQ regression tasks from the Fitness Landscape Inference Proteins (FLIP) benchmark. We compared across different degrees distributional shift using metrics that assess each method’s accuracy, calibration, coverage, width, rank correlation. Additionally, one-hot encoding pretrained language representations, tested retrospective Bayesian settings. Our indicate there no single best method all splits, metrics, uncertainty-based sampling often unable outperform greedy optimization. These benchmarks us provide recommendations more effective design biological machine learning.