Evaluating Dynamic Environment Difficulty for Obstacle Avoidance Benchmarking

• DOI: 10.48550/arxiv.2404.14848
• Publication Date: 2024-04-23
• Authors: Moji Shi; Gang Chen; Álvaro Serra Gómez; Siyuan Wu; Javier Alonso‐Mora
• Keywords: Benchmarking; Obstacle avoidance

Dynamic obstacle avoidance is a popular research topic for autonomous systems, such as micro aerial vehicles and service robots. Accurately evaluating the performance of dynamic methods necessitates establishment metric to quantify environment's difficulty, crucial aspect that remains unexplored. In this paper, we propose four metrics measure difficulty environments. These aim comprehensively capture influence obstacles' number, size, velocity, other factors on difficulty. We compare proposed with existing static environment validate them through over 1.5 million trials in customized simulator. This simulator excludes effects perception control errors supports different motion gaze planners avoidance. The results indicate survivability outperforms establishes monotonic relationship between success rate, Spearman's Rank Correlation Coefficient (SRCC) 0.9. Specifically, every planner, lower leads higher rate. not only facilitates fair comprehensive benchmarking but also provides insights refining collision methods, thereby furthering evolution systems