智能城市应用

A Deep Reinforcement Learning Based Car Following Model for Electric Vehicle

WuYuankai, TanHuachun, PengJiankun, RanBin

摘要


Car following (CF) models are an appealing research area because they fundamentally describe longitudinal interactions of vehicles on the road, and contribute significantly to an understanding of traffic flow. There is an emerging trend to use data-driven method to build CF models. One challenge to the data-driven CF models is their capability to achieve optimal longitudinal driven behavior because a lot of bad driving behaviors will be learnt from human drivers by the supervised learning manner. In this study, by utilizing the deep reinforcement learning (DRL) techniques trust region policy optimization (TRPO), a DRL based CF model for electric vehicle (EV) is built. The proposed CF model can learn optimal driving behavior by itself in simulation. The experiments on following standard driving cycle show that the DRL model outperforms the traditional CF model in terms of electricity consumption.

关键词


autonomous electric vehicle; car following model; deep reinforcement learning; trust region policy optimization

全文:

PDF

参考


Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866.

Chandler, R. E., Herman, R., & Montroll, E. W. (1958). Traffic dynamics: studies in car following. Operations research, 6(2), 165-184.

Chong, L., Abbas, M. M., Flintsch, A. M., & Higgs, B. (2013). A rule-based neural network approach to model driver naturalistic behavior in traffic. Transportation Research Part C: Emerging Technologies, 32, 207-223.

He, Z., Zheng, L., & Guan, W. (2015). A simple nonparametric car-following model driven by field data. Transportation Research Part B: Methodological, 80, 185-201.

Hongfei, J., Zhicai, J., & Anning, N. (2003, October). Develop a car-following model using data collected by" five-wheel system". In Intelligent Transportation Systems, 2003. Proceedings. 2003 IEEE (Vol. 1, pp. 346-351). IEEE.

Huang, X., Sun, J., & Sun, J. (2018). A car-following model considering asymmetric driving behavior based on long short-term memory neural networks. Transportation Research Part C: Emerging Technologies, 95, 346-362.

Khodayari, A., Ghaffari, A., Kazemi, R., & Braunstingl, R. (2012). A modified car-following model based on a neural network model of the human driver effects. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 42(6), 1440-1449.

Liang, K. Y., Mårtensson, J., & Johansson, K. H. (2016). Heavy-duty vehicle platoon formation for fuel efficiency. IEEE Transactions on Intelligent Transportation Systems, 17(4), 1051-1061.

Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., ... & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.

Maerivoet, S., & De Moor, B. (2005). Cellular automata models of road traffic. Physics reports, 419(1), 1-64.

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Petersen, S. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529.

Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... & Kavukcuoglu, K. (2016, June). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928-1937).

Onori, S., Serrao, L., & Rizzoni, G. (2016). Hybrid electric vehicles: Energy management strategies. Berlin Heidelberg: Springer.

Punzo, V., Ciuffo, B., & Montanino, M. (2012). Can results of car-following model calibration based on trajectory data be trusted?. Transportation Research Record, 2315(1), 11-24.

Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017). Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864.

Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, 61, 85-117.

Schulman, J., Levine, S., Abbeel, P., Jordan, M., & Moritz, P. (2015, June). Trust region policy optimization. In International Conference on Machine Learning (pp. 1889-1897).

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., ... & Dieleman, S. (2016). Mastering the game of Go with deep neural networks and tree search. nature, 529(7587), 484.

Wang, X., Jiang, R., Li, L., Lin, Y., Zheng, X., & Wang, F. Y. (2018). Capturing car-following behaviors by deep learning. IEEE Transactions on Intelligent Transportation Systems, 19(3), 910-920.

Zheng, J., Suzuki, K., & Fujita, M. (2013). Car-following behavior with instantaneous driver–vehicle reaction delay: A neural-network-based methodology. Transportation research part C: emerging technologies, 36, 339-351.

Zhou, M., Qu, X., & Li, X. (2017). A recurrent neural network based microscopic car following model to predict traffic oscillation. Transportation research part C: emerging technologies, 84, 245-264.




DOI: https://doi.org/10.33142/sca.v2i5.813

Refbacks

  • 当前没有refback。


版权所有（c）{$ COPYRIGHTYEAR} {$ copyrightHolder}

Creative Commons License
此作品已接受知识共享署名-非商业性使用 4.0国际许可协议的许可。