Multi-Agent Deep Reinforcement Learning for Adaptive Traffic Signal and Vehicle Routing Optimization in VANETs Using SUMO
Keywords:
Adaptive traffic signal control, Reinforcement learning, Traffic conditions, Urban traffic, Vehicle routing, Vehicular ad hoc networks (VANETs)Abstract
Urban traffic congestion is a persistent and escalating challenge in modern cities, leading to increased travel time, fuel consumption, and environmental pollution. Traditional traffic control systems, typically based on pre-timed or actuated signals, lack the adaptability required to respond dynamically to fluctuating traffic conditions. Simultaneously, static vehicle routing approaches fail to consider real-time changes in traffic flow, resulting in inefficient navigation and increased congestion. To address these limitations, this paper proposes an integrated framework that combines Multi-Agent Deep Reinforcement Learning (MADRL) with Vehicular Ad Hoc Networks (VANETs) for intelligent and adaptive traffic signal control and vehicle routing.
In the proposed system, each traffic signal and vehicle operate as an autonomous agent capable of learning optimal policies through interaction with a dynamic urban traffic environment. Agents leverage real-time communication enabled by VANETs specifically Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) protocols to share traffic state information and coordinate actions. The simulation environment is modelled using SUMO (Simulation of Urban Mobility), which accurately replicates urban traffic patterns and supports the deployment of MADRL agents across multiple intersections and vehicles.
The framework uses a decentralized, cooperative learning strategy where traffic signal agents aim to minimize vehicle queue lengths and waiting times by adjusting signal phases in response to live traffic conditions, while vehicle agents continuously update routes based on congestion levels and signal timings. Extensive experiments demonstrate that the proposed approach significantly outperforms conventional static systems and even centralized learning models in terms of average travel time, traffic throughput, and fuel efficiency.
Moreover, the system exhibits robustness in dynamically changing environments and shows promise for scalability across larger traffic networks. However, challenges such as communication overhead, training complexity, and partial observability remain open issues for future exploration. This research lays the groundwork for developing intelligent, adaptive, and cooperative traffic management systems that can be deployed in real-world smart cities to alleviate congestion, reduce emissions, and enhance road safety.
References
A. Hussain, T. Wang, and C. Jiahua, “Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication,” Arxiv.Org, 2020. https://arxiv.org/abs/2002.09853
J. Dinneweth, A. Boubezoul, R. Mandiau, and S. Espié, “Multi-agent reinforcement learning for autonomous vehicles: a survey,” Autonomous Intelligent Systems, vol. 2, no. 1, Nov. 2022, doi: https://doi.org/10.1007/s43684-022-00045-z.
T. Chu, J. Wang, L. Codeca, and Z. Li, “Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–10, 2019, doi: https://doi.org/10.1109/tits.2019.2901791.
Y. Song, H. Zhao, R. Luo, L. Huang, Y. Zhang, and R. Su, “A SUMO Framework for Deep Reinforcement Learning Experiments Solving Electric Vehicle Charging Dispatching Problem,” Arxiv.Org, 2022. https://arxiv.org/abs/2209.02921.
W. Jia and M. Ji, “Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control with Spatio-Temporal Attention Mechanism,” Applied Sciences, vol. 15, no. 15, p. 8605, Aug. 2025, doi: https://doi.org/10.3390/app15158605.
X. Peng, H. Gao, G. Han, H. Wang, and M. Zhang, “Joint Optimization of Traffic Signal Control and Vehicle Routing in Signalized Road Networks using Multi-Agent Deep Reinforcement Learning,” arXiv.org, 2023. https://arxiv.org/abs/2310.10856.
H. Su, Y. D. Zhong, J. Y. J. Chow, B. Dey, and L. Jin, “EMVLight: A multi-agent reinforcement learning framework for an emergency vehicle decentralized routing and traffic signal control system,” Transportation Research Part C: Emerging Technologies, vol. 146, p. 103955, Jan. 2023, doi: https://doi.org/10.1016/j.trc.2022.103955.
S. Wang and S. Wang, “A Novel Multi-Agent Deep RL Approach for Traffic Signal Control,” Arxiv.Org, 2023. https://arxiv.org/abs/2306.02684
F.-X. Devailly, D. Larocque, and L. Charlin, “IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 7496–7507, Jul. 2022, doi: https://doi.org/10.1109/TITS.2021.3070835.
A. Liu, G. Xu, G. Xu, C. Wang, and P. Zuo, “Deep Reinforcement Learning-Based Intelligent Security Forwarding Strategy for VANET,” Sensors, vol. 23, no. 3, pp. 1204–1204, Jan. 2023, doi: https://doi.org/10.3390/s23031204.
T. Hu, Z. Hu, Z. Lu, and X. Wen, “Dynamic traffic signal control using mean field multi‐agent reinforcement learning in large scale road‐networks,” IET Intelligent Transport Systems, vol. 17, no. 9, pp. 1715–1728, Apr. 2023, doi: https://doi.org/10.1049/itr2.12364.
J. Guo, L. Cheng, and S. Wang, “CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles Using Deep Reinforcement Learning,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–12, 2023, doi: https://doi.org/10.1109/TITS.2023.3276416.
R. Bokade and X. Jin, “PyTSC: A Unified Platform for Multi-Agent Reinforcement Learning in Traffic Signal Control,” Sensors, vol. 25, no. 5, p. 1302, Feb. 2025, doi: https://doi.org/10.3390/s25051302.
A. Louw, L. Labuschagne, and T. Woodley, “Comparison of Reinforcement Learning Agents Applied to Traffic Signal Optimisation,” SUMO Conference Proceedings, vol. 3, pp. 15–43, Sep. 2022, doi: https://doi.org/10.52825/scp.v3i.116.
P. Yadav, A. Mishra, and S. Kim, “A Comprehensive Survey on Multi-Agent Reinforcement Learning for Connected and Automated Vehicles,” Sensors, vol. 23, no. 10, pp. 4710–4710, May 2023, doi: https://doi.org/10.3390/s23104710.
A. Mushtaq, I. U. Haq, M. A. Sarwar, A. Khan, W. Khalil, and M. A. Mughal, “Multi-Agent Reinforcement Learning for Traffic Flow Management of Autonomous Vehicles,” Sensors, vol. 23, no. 5, p. 2373, Feb. 2023, doi: https://doi.org/10.3390/s23052373.
A. Zeynivand., “Traffic flow control using multi-agent reinforcement learning,” Journal of Network and Computer Applications, vol. 207, p. 103497, Nov. 2022, doi: https://doi.org/10.1016/j.jnca.2022.103497.
K. Cao, L. Wang, S. Zhang, “Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning,” Electronics, vol. 13, no. 1, pp. 198–198, Jan. 2024, doi: https://doi.org/10.3390/electronics13010198.
J. Ma and F. Wu, “Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control,” Arxiv.Org, 2022. https://arxiv.org/abs/2205.13836
