تشخیص وضعیت لغزندگی جاده با استفاده از تصاویر دوربین‌های جاده‌ایی مبتنی بر شبکه‌های عصبی پیچشی و یادگیری انتقالی

نوع مقاله : مقاله پژوهشی

نویسندگان

1 کارشناسی ارشد، دانشکده مهندسی مکانیک، دانشگاه علم و صنعت ایران، تهران، ایران

2 استاد، دانشکده مهندسی مکانیک، دانشگاه علم و صنعت ایران، تهران، ایران

3 استادیار، دانشکده مهندسی خودرو، دانشگاه علم و صنعت ایران، تهران، ایران

چکیده

تشخیص وضعیت لغزندگی سطح جاده امری مهم در راستای افزایش امنیت جاده و سرنشینان و همچنین توسعه خودروهای خودران و فناوری‌های مرتبط با آن است. در این راستا پژوهش‌های مختلفی با روش‌ها و حسگرهای متفاوت، با استفاده از داده‌های گوناگونی نظیر تصویر، صوت و فرکانس موج صورت گرفته‌است. این مقاله بدون استفاده از حسگرها و روش‌های پرهزینه تنها با استفاده از تصاویر دوربین‌های مداربسته موجود در جاده‌ها و بهره‌گیری از شبکه‌های عصبی پیچشی انجام شده است. ایده اصلی پژوهش جاری استفاده از رویکرد یادگیری انتقالی است. بنابراین در ابتدا اهمیت و مزایای استفاده از یادگیری انتقالی، در قالب آموزش شبکه‌ای با ساختار InceptionNetv3 بیان شده است. در مرحله بعد با استفاده از چارچوبی جدید به نام GFNet، شبکه عصبی پیچشی ResNet50 و شبکه عصبی بازگشتی با یکدیگر ترکیب و با استفاده از یادگیری انتقالی آموزش داده شده‌اند. درنهایت شبکه‌ای با توانایی تشخیص سطح جاده، در سه دسته خشک، خیس و برفی با دقتی بالغ ‌بر 96% به‌دست آمده است.

کلیدواژه‌ها

عنوان مقاله [English]

Detection of Slippery Road Conditions using the Road CCTV Images based on the Convolutional Neural Networks and Transfer Learning

نویسندگان [English]

  • Mohammad Amin Giyahban 1
  • Mohammad H Shojaeefard 2
  • Abdollah Amirkhani 3
1 Master's degree, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran
2 Professor, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran
3 Assistant Professor, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran
چکیده [English]

The detection of slippery road conditions is one of the main factors needed in order to increase the road and passenger safety, as well as the development of autonomous vehicles and related technologies. In this regard, various researches have been done with different methods and sensors, using data in the different forms of image, sound and wave frequencies. In this article, we have detected the slippery road condition without the use of expensive sensors and methods by using CCTV images of the roads and based on convolutional neural networks. The main idea of this research is the use of transfer learning approach. Therefore, first, the importance and benefits of using transfer learning are presented in the form of network training with InceptionNetv3 architecture. In the next step, a ResNet50 CNN and a recurrent neural network are combined using a new framework called GFNet and are trained by using transfer learning. Finally, a tool with the ability to detect the road surface, in three classes of dry, wet and snow, has been obtained with an accuracy of 96.33%.

کلیدواژه‌ها [English]

  • Convolutional Neural Networks
  • Transfer Learning
  • Deep Learning
  • Road Safety
  • Classification

Smiley face

مراجع

[1]     M. Li, Y. Li, and M. Jiang, "Lane Detection Based on Connection of Various Feature Extraction Methods," Advances in Multimedia, vol. 2018, pp. 1-13, 2018.
[2]     Y. Y. Ye, X. L. Hao, and H. J. Chen, "Lane Detection Method Based on Lane Structural Analysis and CNNs," IET Intelligent Transport Systems, vol. 12, no. 6, pp. 513-520, 2018.
[3]     L. Caltagirone, M. Bellone, L. Svensson, and M. Wahde, "LIDAR–camera Fusion for Road Detection Using Fully Convolutional Neural Networks," Robotics and Autonomous Systems, vol. 111, pp. 125-131, 2019.
[4]     B. Varona, A. Monteserin, and A. Teyseyre, "A Deep Learning Approach to Automatic Road Surface Monitoring and Pothole Detection," Personal and Ubiquitous Computing, vol.24, no. 519-534, pp. 1-16, 2019.
[5]      R. Fan, M. J. Bocus, Y. Zhu, J. Jiao, L. Wang, F. Ma, S. Cheng, M. Liu., "Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding," IEEE Intelligent Vehicles Symposium (IV), IEEE, pp. 474-479, 2019. 
[6]      M. D. Jenkins, T. A. Carr, M. I. Iglesias, T. Buggy, and G. Morison, "A Deep Convolutional Neural Network for Semantic Pixel-wise Segmentation of Road and Pavement Surface Cracks,"  26th European Signal Processing Conference (EUSIPCO), IEEE, pp. 2120-2124, 2018. 
[7]     C. Chun and S.-K. Ryu, "Road Surface Damage Detection Using Fully Convolutional Neural Networks and Semi-supervised Learning," Sensors, vol. 19, no. 24, p. 5501, 2019.
[8]     A. Khoshravian, M. M. Tehrani, and A. Amirkhani, "Semantic Segmentation of Autonomous Vehicles Images with TeacherStudent Technique," Journal of Electronical & Cyber Defence, vol. 9, no. 4, pp.1-19, 2021 (In Persian).
[9]      A. Czyżewski, A. Sroczyński, T. Śmiałkowski, and P. Hoffmann, "Development of Intelligent Road Signs with V2X Interface for Adaptive Traffic Controlling," 6th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), IEEE, pp. 1-7, 2019. 
[10]    S. Roychowdhury, M. Zhao, A. Wallin, N. Ohlsson, and M. Jonasson, "Machine Learning Models for Road Surface and Friction Estimation Using Front-camera Images," International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 1-8, 2018. 
[11]    G. Pan, L. Fu, R. Yu, and M. Muresan, "Evaluation of Alternative Pre-trained Convolutional Neural Networks for Winter Road Surface Condition Monitoring," 5th International Conference on Transportation Information and Safety (ICTIS), IEEE, pp. 614-620, 2019. 
[12]   A. Bystrov, E. Hoare, T.-Y. Tran, N. Clarke, M. Gashinova, and M. Cherniakov, "Automotive System for Remote Surface Classification," Sensors, vol. 17, no. 4, pp. 745, 2017.
[13]   M. Budzyński and A. Tubis, "Assessing the Effects of the Road Surface and Weather Conditions on Road Safety," Journal of KONBiN, vol. 49, no. 3, pp. 323-349, 2019.
[14]   E. Ackerman, "Fatal Tesla Self-driving Car Crash Reminds Us that Robots Aren’t Perfect," IEEE-Spectrum, vol. 1, 2016.
[15]   M. R. Endsley, "Autonomous Driving Systems: A Preliminary Naturalistic Study of the Tesla Model S," Journal of Cognitive Engineering and Decision Making, vol. 11, no. 3, pp. 225-238, 2017.
[16]   A. Taeihagh and H. S. M. Lim, "Governing Autonomous Vehicles: Emerging Responses for Safety, Liability, Privacy, Cybersecurity, and Industry Risks," Transport reviews, vol. 39, no. 1, pp. 103-128, 2019.
[17]   A. Alnasser, H. Sun, and J. Jiang, "Cyber Security Challenges and Solutions for V2X Communications: A Survey," Computer Networks, vol. 151, pp. 52-67, 2019.
[18]   S. N. Doustimotlagh, "A New Framework for Enhancing the Security of Military Internet of Things Using the Hybrid Classical-Quantum Cryptography," Journal of Electronical & Cyber Defence, vol. 9, no. 2, pp. 29-49, 2021 (In Persian).
[19]   R. Robinson, "Slippery Road Detection and Evaluation," University of Michigan, Ann Arbor, Transportation Research Institute, 2012. 
[20]   D. Grabowski and A. Czyżewski, "System for Monitoring Road Slippery Based on CCTV Cameras and Convolutional Neural Networks," Journal of Intelligent Information Systems, vol. 55, no. 3, pp. 521-534, 2020.
[21]   W. Kongrattanaprasert, H. Nomura, T. Kamakura, and K. Ueda, "Detection of Road Surface States from Tire Noise Using Neural Network Analysis," IEEJ Transactions on Industry Applications, vol. 130, no. 7, pp. 920-925, 2010.
[22]   D. Lee, J.-C. Kim, M. Kim, and H. Lee, "Intelligent Tire Sensor-Based Real-Time Road Surface Classification Using an Artificial Neural Network," Sensors, vol. 21, no. 9, p. 3233, 2021.
[23]   J. Casselgren, M. Kutila, and M. Jokela, "Slippery Road Detection by Using Different Methods of Polarised Light," In Advanced Microsystems for Automotive Applications,  Springer, pp. 207-220, 2012.
[24]    V. Viikari, T. Varpula, and M. Kantanen, "Automotive Radar Technology for Detecting Road Conditions. Backscattering Properties of Dry, Wet, and Icy Asphalt," European Radar Conference, IEEE, pp. 276-279, 2008. 
[25]    P. Jonsson, "Remote Sensor for Winter Road Surface Status Detection," SENSORS, 2011 IEEE, pp. 1285-1288, 2011. 
[26]   Y. Kim, N. Baik, and J. Kim, "A Study on Development of Mobile Road Surface Condition Detection System Utilizing Probe Car," Journal of emerging trends in computing and information sciences, vol. 4, no. 10, pp. 742-750, 2013.
[27]   J. Alonso, J. M. López, I. Pavón, M. Recuero, C. Asensio, G. Arcas, A. Bravo, "On-board Wet Road Surface Identification Using Tyre/Road Noise and Support Vector Machines," Applied acoustics, vol. 76, pp. 407-415, 2014.
[28]    M. Nolte, N. Kister, and M. Maurer, "Assessment of Deep Convolutional Neural Networks for Road Surface Classification," 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE, pp. 381-386, 2018. 
[29]   Y. Wang, K. Lv, R. Huang, S. Song, L. Yang, and G. Huang, "Glance and Focus: A Dynamic Approach to Reducing Spatial Redundancy in Image Classification," Presented at the NeurIPS, 2020.
[30]   S. P. Jakhar, A. Nandal, and R. Dixit, "Classification and Measuring Accuracy of Lenses Using Inception Model V3," In Innovations in Computational Intelligence and Computer Vision: Springer, pp. 376-383, 2021.
[31]    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the Inception Architecture for Computer Vision," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016.
[32]    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the Inception Architecture for Computer Vision," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016. 
[33]    A. Mittal, A. K. Moorthy, and A. C. Bovik, "Blind/Referenceless Image Spatial Quality Evaluator," Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR) IEEE, pp. 723-727, 2011.
پدافند الکترونیکی و سایبری
دوره 10، شماره 2 - شماره پیاپی 38
شماره پیاپی 38، فصلنامه تابستان
مهر 1401
صفحه 105-116

فایل ها

سابقه مقاله

  • تاریخ دریافت: 17 مرداد 1400
  • تاریخ بازنگری: 08 بهمن 1400
  • تاریخ پذیرش: 18 مرداد 1401
  • تاریخ انتشار: 01 مهر 1401

هم رسانی

ارجاع به این مقاله

آمار