Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners

• DOI: 10.48550/arxiv.2307.01928
• Publication Date: 2023-01-01
• Authors: Allen Z. Ren; Anushri Dixit; A. A. Bodrova; Sumeet Singh; Stephen Tu; Noah Brown; Peng Xu; Leila Takayama; Fei Xia; Jake Varley; Zhenjia Xu; Dorsa Sadigh; Andy Zeng; Anirudha Majumdar
• Keywords: Hallucinating; Ask price; Complement; Conflation

Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning that may provide utility for robots, but remain prone confidently hallucinated predictions. In this work, we present KnowNo, which is framework measuring and aligning the uncertainty LLM-based planners such they know when don't ask help needed. KnowNo builds on theory conformal prediction statistical guarantees task completion while minimizing human in complex multi-step settings. Experiments across variety simulated real robot setups involve tasks with different modes ambiguity (e.g., spatial numeric uncertainties, preferences Winograd schemas) show performs favorably over modern baselines (which ensembles or extensive prompt tuning) terms improving efficiency autonomy, providing formal assurances. can be used LLMs out box without model-finetuning, suggests lightweight approach modeling complement scale growing foundation models. Website: https://robot-help.github.io