CycLight: Learning traffic signal cooperation with a cycle-level strategy

• DOI: 10.1016/j.eswa.2024.124543
• Publication Date: 2024-06-22T15:56:03Z
• Authors: Gengyue Han; Xiaohan Liu; Yu Han; Xianyue Peng; Hao Wang
• Keywords: Computer Science - Machine Learning; Computer Science - Artificial Intelligence; 0211 other engineering and technologies; 02 engineering and technology; Electrical Engineering and Systems Science - Systems and Control

This study introduces CycLight, a novel cycle-level deep reinforcement learning (RL) approach for network-level adaptive traffic signal control (NATSC) systems. Unlike most traditional RL-based traffic controllers that focus on step-by-step decision making, CycLight adopts a cycle-level strategy, optimizing cycle length and splits simultaneously using Parameterized Deep Q-Networks (PDQN) algorithm. This cycle-level approach effectively reduces the computational burden associated with frequent data communication, meanwhile enhancing the practicality and safety of real-world applications. A decentralized framework is formulated for multi-agent cooperation, while attention mechanism is integrated to accurately assess the impact of the surroundings on the current intersection. CycLight is tested in a large synthetic traffic grid using the microscopic traffic simulation tool, SUMO. Experimental results not only demonstrate the superiority of CycLight over other state-of-the-art approaches but also showcase its robustness against information transmission delays.