General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning

• DOI: 10.1016/j.trc.2023.104367
• Publication Date: 2023-10-10T23:28:52Z
• Authors: Yutong Chen; Yan Xu; Lei Yang; Minghua Hu
• Keywords: Separation (statistics); Air Traffic Management

Reinforcement learning (RL) techniques have been studied for solving the conflict resolution (CR) problem in air traffic management, leveraging their potential computation and ability to handle uncertainty. However, challenges remain that impede application of RL methods CR practice, including three-dimensional manoeuvres, generalisation, trajectory recovery, success rate. This paper proposes a general multi-agent reinforcement approach real-time multi-aircraft resolution, which agents share neural network are deployed on each aircraft form distributed decision-making system. To address challenges, several technologies introduced, partial observation model based imminent threats safety separation relaxation multiple flight levels an adaptive manoeuvre strategy buffer The Rainbow Deep Q-learning Network (DQN) is used enhance efficiency process. A simulation environment considers uncertainty (resulting from mechanical navigation errors wind) constructed train evaluate proposed approach. experimental results demonstrate method can resolve conflicts scenarios with much higher density than today's real-world situations.