Real-world ride-hailing vehicle repositioning using deep reinforcement learning

• DOI: 10.1016/j.trc.2021.103289
• Publication Date: 2021-07-22T23:23:11Z
• Authors: Yan Jiao; Xiaocheng Tang; Zhiwei (Tony) Qin; Shuaiji Li; Fan Zhang; Hongtu Zhu; Jieping Ye
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Artificial Intelligence; 05 social sciences; 02 engineering and technology; Machine Learning (cs.LG); Artificial Intelligence (cs.AI); 0502 economics and business; 0202 electrical engineering, electronic engineering, information engineering; Computer Science - Multiagent Systems; Multiagent Systems (cs.MA)

We present a new practical framework based on deep reinforcement learning and decision-time planning for real-world vehicle repositioning on ride-hailing (a type of mobility-on-demand, MoD) platforms. Our approach learns the spatiotemporal state-value function using a batch training algorithm with deep value networks. The optimal repositioning action is generated on-demand through value-based policy search, which combines planning and bootstrapping with the value networks. For the large-fleet problems, we develop several algorithmic features that we incorporate into our framework and that we demonstrate to induce coordination among the algorithmically-guided vehicles. We benchmark our algorithm with baselines in a ride-hailing simulation environment to demonstrate its superiority in improving income efficiency meausred by income-per-hour. We have also designed and run a real-world experiment program with regular drivers on a major ride-hailing platform. We have observed significantly positive results on key metrics comparing our method with experienced drivers who performed idle-time repositioning based on their own expertise.