Traffic Signal Control of a Crossroad Using Reinforcement Learning Methods (Q-Learning, Sarsa, Eligibility Traces)

Document Type : Original Article

Authors

1 M.Sc. Student, Electrical and Biomedical Faculty, Sadjad University of Technology, Mashhad, Iran

2 Ph.D. Student, Electrical and Biomedical Faculty, Sadjad University of Technology, Mashhad, Iran.

Abstract

One of the most important goals of research in the field of transportation is optimizing of traffic flows. Today there are many problems in traffic flows such as continuous growth of vehicles, the limitation in the resources provided by the current infrastructure and the nonlinear, dynamic and random nature of the traffic flow. For solving this problem, use of intelligent methods in controlling the flows of traffic, especially the methods of reinforcement learning is investigated. In addition to simplicity and lack of computational complexity, the learning procedure is model free that is there is no need of a mathematical model. Other advantages of this method are the ability to adapt to environmental conditions and robustness to environmental changes. In this paper, traffic control of an intersection is carried out using three methods of reinforcement learning (Q-learning, SARSA, and Eligibility traces). Simulation results indicate that eligibility traces method is more efficient than the two other methods of Q-learning and Sarsa, which has been used previously in traffic control articles.
 
 
 

Keywords

References

Chu, T., Qu, S., & Wang, J. (2016). "Large-scale traffic grid signal control with regional Reinforcement Learning". In American Control Conference (ACC), pp. 815-820.
 
-Dusparic, I., Monteil, J., & Cahill, V. (2016), "Towards autonomic urban traffic control with collaborative multi-policy reinforcement learning". In Intelligent Transportation Systems (ITSC), IEEE 19th International Conference on, pp. 2065-2070.
 
-Prabuchandran, K. J., AN, H. K., & Bhatnagar, S. (2014), "Multi-agent reinforcement learning for traffic signal control. In Intelligent Transportation Systems (ITSC)", IEEE 17th International Conference on, pp. 2529-2534.
 
-Wu, L., Zhang, X., & Shi, Z. (2010). "An intelligent fuzzy control for crossroads traffic light". In Intelligent systems (GCIS), second WRI global congress on, Vol. 3, pp. 28-32.
 
-Odeh, S. M. (2015), "Hybrid algorithm: fuzzy logic-genetic algorithm on traffic light intelligent system". In Fuzzy Systems
(FUZZ-IEEE), 2015 IEEE International Conference on, pp.1-7.
 
-Abhishek, K., & Misra, B. B. (2016), “Hybrid Genetic Algorithm and time delay neural network model for Forecasting Traffic flow”. In Engineering and Technology (ICETECH), IEEE International Conference on,
pp. 178-183.
-Chang-sheng, Z., Jian-bo, L., Wen-yi, F., & Xu-gang, M. (2010), “Intelligent dispatch for public traffic vehicles based on improved Genetic –Algorithm”. In Future Computer and Communication (ICFCC), 2nd International Conference on, Vol. 3, pp. V3-372.
 
-Kim, S. (1994), "Application of Petri Networks and Fuzzy Logic to Advanced Traffic Management Systems".          
Ph.D Thesis, Polytechnic University,
USA, 139 P.
 
-Qiao, J., Yang, N., & Gao, J. (2011),
"Two-stage fuzzy logic controller for signalized intersection". IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(1), pp.178-184.
 
-Liu, Z. (2007), "A survey of intelligence methods in urban traffic signal control". IJCSNS International Journal of Computer Science and Network Security, 7(7), pp.105-112.
 
-Park, B., Messer, C. and Urbanik II, T. (2000), "Enhanced genetic algorithm for signal-timing   optimization of oversaturated intersections” Transportation Research Record: Journal of the Transportation Research Board, (1727), pp.32-41.
 
-Xu Dongling, et al. (1992), "A Fuzzy Controller of Traffic Systems and Its Neural Network Implementation". Information and Control, 21(2): pp. 74-78.
 
-Tian, Y., Li, Z., Zhou, D., Song, J., & Xiao, D. (2008), "Interactive signal control for over-saturated arterial intersections using fuzzy logic". In Intelligent Transportation Systems,. ITSC. 11th International IEEE Conference on, pp. 1067-1072.
 
-Sutton RS, Barto AG.(1998), "Reinforcement learning: An introduction". Cambridge: MIT press.
 

Files

Share

How to cite

Statistics