Detection of Slippery Road Conditions using the Road CCTV Images based on the Convolutional Neural Networks and Transfer Learning

Document Type : Original Article

Authors

1 Master's degree, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran

2 Professor, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran

3 Assistant Professor, Faculty of Mechanical Engineering, Iran University of Science and Technology, Tehran, Iran

Abstract

The detection of slippery road conditions is one of the main factors needed in order to increase the road and passenger safety, as well as the development of autonomous vehicles and related technologies. In this regard, various researches have been done with different methods and sensors, using data in the different forms of image, sound and wave frequencies. In this article, we have detected the slippery road condition without the use of expensive sensors and methods by using CCTV images of the roads and based on convolutional neural networks. The main idea of this research is the use of transfer learning approach. Therefore, first, the importance and benefits of using transfer learning are presented in the form of network training with InceptionNetv3 architecture. In the next step, a ResNet50 CNN and a recurrent neural network are combined using a new framework called GFNet and are trained by using transfer learning. Finally, a tool with the ability to detect the road surface, in three classes of dry, wet and snow, has been obtained with an accuracy of 96.33%.

Keywords

Smiley face

References

[1]     M. Li, Y. Li, and M. Jiang, "Lane Detection Based on Connection of Various Feature Extraction Methods," Advances in Multimedia, vol. 2018, pp. 1-13, 2018.
[2]     Y. Y. Ye, X. L. Hao, and H. J. Chen, "Lane Detection Method Based on Lane Structural Analysis and CNNs," IET Intelligent Transport Systems, vol. 12, no. 6, pp. 513-520, 2018.
[3]     L. Caltagirone, M. Bellone, L. Svensson, and M. Wahde, "LIDAR–camera Fusion for Road Detection Using Fully Convolutional Neural Networks," Robotics and Autonomous Systems, vol. 111, pp. 125-131, 2019.
[4]     B. Varona, A. Monteserin, and A. Teyseyre, "A Deep Learning Approach to Automatic Road Surface Monitoring and Pothole Detection," Personal and Ubiquitous Computing, vol.24, no. 519-534, pp. 1-16, 2019.
[5]      R. Fan, M. J. Bocus, Y. Zhu, J. Jiao, L. Wang, F. Ma, S. Cheng, M. Liu., "Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding," IEEE Intelligent Vehicles Symposium (IV), IEEE, pp. 474-479, 2019. 
[6]      M. D. Jenkins, T. A. Carr, M. I. Iglesias, T. Buggy, and G. Morison, "A Deep Convolutional Neural Network for Semantic Pixel-wise Segmentation of Road and Pavement Surface Cracks,"  26th European Signal Processing Conference (EUSIPCO), IEEE, pp. 2120-2124, 2018. 
[7]     C. Chun and S.-K. Ryu, "Road Surface Damage Detection Using Fully Convolutional Neural Networks and Semi-supervised Learning," Sensors, vol. 19, no. 24, p. 5501, 2019.
[8]     A. Khoshravian, M. M. Tehrani, and A. Amirkhani, "Semantic Segmentation of Autonomous Vehicles Images with TeacherStudent Technique," Journal of Electronical & Cyber Defence, vol. 9, no. 4, pp.1-19, 2021 (In Persian).
[9]      A. Czyżewski, A. Sroczyński, T. Śmiałkowski, and P. Hoffmann, "Development of Intelligent Road Signs with V2X Interface for Adaptive Traffic Controlling," 6th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), IEEE, pp. 1-7, 2019. 
[10]    S. Roychowdhury, M. Zhao, A. Wallin, N. Ohlsson, and M. Jonasson, "Machine Learning Models for Road Surface and Friction Estimation Using Front-camera Images," International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 1-8, 2018. 
[11]    G. Pan, L. Fu, R. Yu, and M. Muresan, "Evaluation of Alternative Pre-trained Convolutional Neural Networks for Winter Road Surface Condition Monitoring," 5th International Conference on Transportation Information and Safety (ICTIS), IEEE, pp. 614-620, 2019. 
[12]   A. Bystrov, E. Hoare, T.-Y. Tran, N. Clarke, M. Gashinova, and M. Cherniakov, "Automotive System for Remote Surface Classification," Sensors, vol. 17, no. 4, pp. 745, 2017.
[13]   M. Budzyński and A. Tubis, "Assessing the Effects of the Road Surface and Weather Conditions on Road Safety," Journal of KONBiN, vol. 49, no. 3, pp. 323-349, 2019.
[14]   E. Ackerman, "Fatal Tesla Self-driving Car Crash Reminds Us that Robots Aren’t Perfect," IEEE-Spectrum, vol. 1, 2016.
[15]   M. R. Endsley, "Autonomous Driving Systems: A Preliminary Naturalistic Study of the Tesla Model S," Journal of Cognitive Engineering and Decision Making, vol. 11, no. 3, pp. 225-238, 2017.
[16]   A. Taeihagh and H. S. M. Lim, "Governing Autonomous Vehicles: Emerging Responses for Safety, Liability, Privacy, Cybersecurity, and Industry Risks," Transport reviews, vol. 39, no. 1, pp. 103-128, 2019.
[17]   A. Alnasser, H. Sun, and J. Jiang, "Cyber Security Challenges and Solutions for V2X Communications: A Survey," Computer Networks, vol. 151, pp. 52-67, 2019.
[18]   S. N. Doustimotlagh, "A New Framework for Enhancing the Security of Military Internet of Things Using the Hybrid Classical-Quantum Cryptography," Journal of Electronical & Cyber Defence, vol. 9, no. 2, pp. 29-49, 2021 (In Persian).
[19]   R. Robinson, "Slippery Road Detection and Evaluation," University of Michigan, Ann Arbor, Transportation Research Institute, 2012. 
[20]   D. Grabowski and A. Czyżewski, "System for Monitoring Road Slippery Based on CCTV Cameras and Convolutional Neural Networks," Journal of Intelligent Information Systems, vol. 55, no. 3, pp. 521-534, 2020.
[21]   W. Kongrattanaprasert, H. Nomura, T. Kamakura, and K. Ueda, "Detection of Road Surface States from Tire Noise Using Neural Network Analysis," IEEJ Transactions on Industry Applications, vol. 130, no. 7, pp. 920-925, 2010.
[22]   D. Lee, J.-C. Kim, M. Kim, and H. Lee, "Intelligent Tire Sensor-Based Real-Time Road Surface Classification Using an Artificial Neural Network," Sensors, vol. 21, no. 9, p. 3233, 2021.
[23]   J. Casselgren, M. Kutila, and M. Jokela, "Slippery Road Detection by Using Different Methods of Polarised Light," In Advanced Microsystems for Automotive Applications,  Springer, pp. 207-220, 2012.
[24]    V. Viikari, T. Varpula, and M. Kantanen, "Automotive Radar Technology for Detecting Road Conditions. Backscattering Properties of Dry, Wet, and Icy Asphalt," European Radar Conference, IEEE, pp. 276-279, 2008. 
[25]    P. Jonsson, "Remote Sensor for Winter Road Surface Status Detection," SENSORS, 2011 IEEE, pp. 1285-1288, 2011. 
[26]   Y. Kim, N. Baik, and J. Kim, "A Study on Development of Mobile Road Surface Condition Detection System Utilizing Probe Car," Journal of emerging trends in computing and information sciences, vol. 4, no. 10, pp. 742-750, 2013.
[27]   J. Alonso, J. M. López, I. Pavón, M. Recuero, C. Asensio, G. Arcas, A. Bravo, "On-board Wet Road Surface Identification Using Tyre/Road Noise and Support Vector Machines," Applied acoustics, vol. 76, pp. 407-415, 2014.
[28]    M. Nolte, N. Kister, and M. Maurer, "Assessment of Deep Convolutional Neural Networks for Road Surface Classification," 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE, pp. 381-386, 2018. 
[29]   Y. Wang, K. Lv, R. Huang, S. Song, L. Yang, and G. Huang, "Glance and Focus: A Dynamic Approach to Reducing Spatial Redundancy in Image Classification," Presented at the NeurIPS, 2020.
[30]   S. P. Jakhar, A. Nandal, and R. Dixit, "Classification and Measuring Accuracy of Lenses Using Inception Model V3," In Innovations in Computational Intelligence and Computer Vision: Springer, pp. 376-383, 2021.
[31]    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the Inception Architecture for Computer Vision," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016.
[32]    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the Inception Architecture for Computer Vision," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016. 
[33]    A. Mittal, A. K. Moorthy, and A. C. Bovik, "Blind/Referenceless Image Spatial Quality Evaluator," Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR) IEEE, pp. 723-727, 2011.
Electronic and Cyber Defense
Volume 10, Issue 2 - Serial Number 38
October 2022
Pages 105-116

Files

History

  • Receive Date: 08 August 2021
  • Revise Date: 28 January 2022
  • Accept Date: 09 August 2022
  • Publish Date: 23 September 2022

Share

How to cite

Statistics