Mechanical Engineering Science

A Review of Lane Detection Based on Deep Learning Methods

ZHANGYunzuo (School of Information science and Technology, Shijiazhuang Tiedao University), TUZhiwei (School of Information science and Technology, Shijiazhuang Tiedao University), LYUFenfen (Sports Technical School of Hebei Sports Bureau)

Abstract


Lane detection is an important aspect of autonomous driving, aiming to ensure that vehicles accurately understand road structures as well as improve their ability to drive in complex traffic environments. In recent years, lane detection tasks based on deep learning methods have made significant progress in detection accuracy. In this paper, we provide a comprehensive review of deep learning-based lane detection tasks in recent years. First, we introduce the background of the lane detection task, including lane detection, the lane datasets and the factors affecting lane detection. Second, we review the traditional and deep learning methods for lane detection, and analyze their features in detail while classifying the different methods. In the deep learning methods classification section, we explore five main categories, including segmentation-based, object detection, parametric curves, end-to-end, and keypoint-based methods. Then, some typical models are briefly compared and analyzed. Finally, in this paper, based on the comprehensive consideration of current lane detection methods, we put forward the current problems still faced, such as model generalization and computational cost. At the same time, possible future research directions are given for extreme scenarios, model generalization and other issues.

Keywords


Deep learning; Lane detection; Image segmentation; Object detection; Parametric curves

Full Text:

PDF

References


Song Ruitao, Li Bin. Surrounding Vehicles’lane change maneuver prediction and detection for intelligent vehicles: a comprehensive review[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23:6046-6062.

Wang Zhangu, Jun Zhan, C. Duan, et al.. A review of vehicle detection techniques for intelligent vehicles[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023,34:3811-3831.

Lin Lianghao. Research on structured road recognition algorithm based on image processing[D]. Harbin University of Science and Technology, 2021.

Zhong Zebin. An image graying method for lane line recognition[J]. Journal of Tongji University (Natural Science Edition), 2019,47:178-182.

Ahmed Hashem, T. Schlechter. Feature-based lane detection algorithms for track following: a comparative study[C]. 2022 International Conference on Mechatronics and Robotics Engineering, 2022.

Yu Fuxing, Wu Yafeng, Suo Yina, et al.. Shallow detail and semantic segmentation combined bilateral network model for lane detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2023,24:8617-8627.

Guo Rui. Research and exploration of lane line detection based on deep learning[C]. International Conference on Image Processing & Computer Vision (IPCV), 2023.

Zhong Haomai. Research and application of lane line detection method based on deep learning[D]. Xi'an University of Technology, 2023.

Tian Sheng, Zhang Jianfeng, Zhang Yutian, et al.. Lane line detection algorithm based on expanded convolutional pyramid network[D]. Journal of Southwest Jiaotong University, 2020,55:386-392.

Li Xiaoxiao, X. Hu, Z. Wang. Advances in instance segmentation research based on deep learning[J]. Computer Engineering and Applications, 2021,57:60-67.

X. Liu, Hu Xiaoguang, Wang Ziqiang, et al.. Progress of anchor-less frame-based object detection method and its application in complex scenes[J]. Journal of Automation, 2023,49:1369-1392.

Yeongmin Ko, Jun Jiwon, Donghwuy Ko, et al.. Key points estimation and point instance segmentation approach for lane detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2021,23:8949-8958.

Lv Chuan. Research on lane line recognition and tracking method based on key point detection[D]. Northwest A&F University, 2021.

Wang Jinsheng, Ma Yinchao, Huang Shaofei, et al.. A keypoint-based global association network for lane detection[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

S. Ettinger, Chen Shuyang , Benjamin Caine, Large scale interactive motion forecasting for autonomous driving: the waymo open motion dataset[C]. 2021 IEEE/CVF International Conference on Computer Vision, 2021.

Zhang Youcheng, Lu Zongqing, Zhang Xuechen, et al.. Deep learning in lane marking detection: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2022,23:5976-5992.

Wang Xi. Lane line detection algorithm in abnormal driving environment[D]. Beijing Jiaotong University, 2022.

L. Tabelini, R. Berriel, T. Paixao. Polylanenet: lane estimation via deep polynomial regression[C]. Proceedings of 2020 25th International Conference on Pattern Recognition, 2020.

Teng Zhu, J. Kim, D. Kang. Real-time lane detection by using multiple cues[C]. International Conference on Control, Automation and Systems, 2010.

S. Kumar, M. Jailia, S. Varshney. An efficient approach for highway lane detection based on the hough transform and kalman filter[J]. Innovative infrastructure solutions, 2022,7(5):290.

G. Taubel, R. Sharma, J. Yang. An experimental study of a lane departure warning system based on the optical flow and hough transform methods [J]. WSEAS Transaction on Systems, 2014,13:105.

W. Kruger, W. Enkelmann, S. Rossle. Real-time estimation and tracking of optical flow vectors for obstacle detection [C].Proceedings of the Intelligent Vehicles' 95. Symposium, 1995.

J. Niu, J. Lu, M. Xu, et al.. Robust lane detection using two-stage feature extraction with curve fitting[J]. Pattern Recognition, 2016,59:225-233.

W. Van Gansbeke, B. Brabandere, D. Neven. End-to-end lane detection through differentiable least-squares fitting[C]. IEEE/CVF International Conference on Computer Vision Workshops, 2019.

A. Krizhevsky, X. Sutskever, G. Hinton. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2012,60:84-90.

S. Yalabaka, C. Prasad, P. Sanjana. Lane detection using deep learning techniques[c]. Proceedings of 2022 8th International Conference on Signal Processing and Communication (ICSC), 2022.

J. Redmon, S. Divvala, R. Girshick. You only look once: unified, real-time object detection[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

Ren Shaoqin, He Kaiming, R. Girshick, et al.. Faster r-cnn: towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2015,39:1137-1149.

A. Vaswani , N. Shazeer, N. Parmar. Attention is all you need[J]. Neural Information Processing Systems, 2017: 5998-6008.

Wang Biyao, Han Yi, Cui Hangbin, et al.. Image-based road semantic segmentation detection method[J]. Journal of Shandong University, 2023,53:37-47.

K. Muhammad, T. Hussain, H. Ullah. Vision-based semantic segmentation in scene understanding for autonomous driving: recent achievements, challenges, and outlooks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022(23):22694-22715.

Tian Xuan, Wang Liang, Ding Qi. A review of deep learning based image semantic segmentation methods[J]. Journal of Software, 2019,30:440-468.

E. Shelhamer, J. Long, T. Darrell. Fully convolutional networks for semantic segmentation[C]. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

V. Badrinarayanan, A. Kendall, R. Cipolla. Segnet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017,39:2481-2495.

H. Noh, S. Hong, B. Han. Learning deconvolution network for semantic segmentation[C]. Proceedings of the IEEE international conference on computer vision, 2015.

O. Ronneberger, P. Fischer, T. Brox. U-Net: convolutional networks for biomedical image segmentation[J]. Proceedings of Medical Image Computing and Computer-Assisted Intervention, 2015:234-241.

Yang Shuqin, Ma Yuhao, Fang Mingyu, et al.. Instance segmentation based lane line detection for complex environments[J]. Journal of Zhejiang University (Engineering Edition), 2022,56:809-815.

A. Paszke, A. Chaurasia,S. Kim. Enet: a deep neural network architecture for real-time semantic segmentation[J]. arXiv, 2016,1606.02147.

Wang Ze, Ren Weiqiang. Qiu Qiang. Lanenet: real-time lane detection networks for autonomous driving[J]. arXiv, 2018,1807.01726.

Yi. S, Jun Yu Li, Xu Xin. Adaptive multi-lane detection based on robust instance segmentation for intelligent vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2022,8:888-899.

Xu Pengbin, Qu Anguo, Wang Kunfeng. A review of panoramic segmentation research[J]. Journal of Automation, 2021,47: 549-568.

Wang Hai, Qiu Meng, Cai Yingfeng, et al.. Sparse u-pdp: a unified multi-task framework for panoptic driving perception[J]. IEEE Transactions on Intelligent Transportation Systems, 2023,24:11308-11320.

Zhao Hengshuang, Shi Jianpin, Qi Xiaojuam, et al.. Pyramid scene parsing network[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.

W. Liu, D. Anguelov, D. Erhan. Ssd: single shot multibox detector[C]. European Conference on Computer Vision (ECCV), 2016.

Tan Mingxing, Pang Ruoming, Q. Le. Efficientdet: scalable and efficient object detection[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020.

He Kaiming, G. Gkioxar, P. Dollár. Mask r-cnn. Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV)[C]. 2017.

Zheng Tu, Huang Yifei, Liu Yang, et al.. Clrnet: cross layer refinement network for lane detection[C]. Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.

Ran Hao, Yin Yunfei, Huang Faliang, et al.. Flamnet: a flexible line anchor mechanism network for lane detection[C]. IEEE Transactions on Intelligent Transportation Systems, 2021.

Yu Zichen, Liu Quanli, Wang Wen, et al.. Dalnet: A rail detection network based on dynamic anchor line[J]. arXiv, 2023,2308.11381.

Qin Zequn, Zhang Pengyi, Li Xi. Ultra fast deep lane detection with hybrid anchor driven ordinal classification[J]. IEEE transactions on pattern analysis and machine intelligence, 2022:1-14.

A. Gurghian, T. Koduri, S. Bailur. Deeplanes: end-to-end lane position estimation using deep neural networks[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016.

D. Neven, B. De, S. Georgoulis. Towards end-to-end lane detection: an instance segmentation approach[C]. 2018 IEEE intelligent vehicles symposium(IV), 2018.

Liu Tuijin, Yuan Zejian, Liu Tie, et al.. End-to-end lane shape prediction with transformers[C]. Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021.

Feng Zhengyang, Guo Shaohua, Tan Xin, et al.. Rethinking efficient lane detection via curve modeling[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.

Z. Qu, Jin Huan, Zhou Yang, et al.. Focus on local: detecting lane marker from bottom up via key point[C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.

Wang Jinsheng, Ma Yinchao, Huang Shaofei. A keypoint-based global association network for lane detection[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.




DOI: https://doi.org/10.33142/mes.v5i2.12721

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Yunzuo ZHANG, Zhiwei TU, Fenfen LYU

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.