Improved POMDP Tree Search Planning with Prioritized Action Branching

• DOI: 10.1609/aaai.v35i13.17412
• Publication Date: 2022-09-08T19:52:24Z
• Authors: John Mern; Anil Yildiz; Lawrence Bush; Tapan Mukerji; Mykel J. Kochende...
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; 0209 industrial biotechnology; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; I.2.8; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Machine Learning (cs.LG)

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. This paper proposes a method called PA-POMCPOW to sample a subset of the action space that provides varying mixtures of exploitation and exploration for inclusion in a search tree. The proposed method first evaluates the action space according to a score function that is a linear combination of expected reward and expected information gain. The actions with the highest score are then added to the search tree during tree expansion. Experiments show that PA-POMCPOW is able to outperform existing state-of-the-art solvers on problems with large discrete action spaces.