Forgery detection in digital images using the hybrid deep learning method

Document Type : Original Article

Authors

1 Assistant Professor, Maybod University, Yazd, Iran.

2 Associate Professor, Yazd University, Yazd, Iran

3 Associate Professor, Maybod University, Yazd, Iran.

Abstract

Today, images are used as powerful communication tools and sources of information. In certain applications, such as medicine, justice, and forensics, images serve as evidence. Therefore, the validity of an image is crucial. With the spread and availability of image editing tools, people can easily manipulate images to their advantage. They follow political, cultural, economic, and social issues by adding or removing elements from images, often distributing misinformation. Consequently, forgery detection is one of the most important and challenging topics in the field of computer vision. This research aims to identify forgery and healthy images and pixels using a hybrid deep learning network. In the proposed method, three pre-trained networks—VGG16, MobileNet, and EfficientNetB0—are employed in three different branches. To detect forgery at both the image and pixel levels, the output feature maps from these branches are merged in a concatenate layer. Subsequently, a global average pooling layer and a scoring layer are used to identify forgery and healthy images. Additionally, feature maps combined from the three branches are utilized to create a heat map image for forgery detection. Notably, pixel forgery detection is performed solely using the heat map image generated from the combined network, without relying on ground truth images that specify the forgery area during training. The proposed method is evaluated on the well-known CoMoFod dataset, demonstrating satisfactory performance against forgery images with various geometric transformations and post-processing operations
 

Keywords

References

  1. B. Abd Warif, A. W. A. Wahab, M. Y. I. Idris, R. Ramli, R., Salleh, S. Shamshirband, & K. K. R. Choo, “Copy-move forgery detection: survey, challenges and future directions,” Journal of Network and Computer Applications, vol. 75, pp. 259-278, 2016, doi: g/10.1016/j.jnca.2016.09.008.
  2. J. Barad, & M. M. Goswami, “Image forgery detection using deep learning: a survey,” In 6th International Conference on Advanced Computing and Communication Systems (ICACCS), pp. 571–576, 2020, doi: 10.1109/ICACCS48705.2020.9074408.
  3. H. Saber, M. A. Khan, & B. G. Mejbel, “A survey on image forgery detection using different forensic approaches,” Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 3, pp. 361–370, 2020, doi:  10.25046/aj050347.
  4. Zhang, C. Wang, & X. Zhou, “A survey on passive image copy-move forgery detection,” Journal of Information Processing Systems, vol. 14, no. 1, pp. 6-31, 2018, doi: 10.3745/JIPS.02.0078.
  5. A. Zedan, M. M. Soliman, K. M. Elsayed, & H. M. Onsi, “Copy move forgery detection techniques: a comprehensive survey of challenges and future directions,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 7, 2021,
  6. Shwetha, & S. V. Sathyanarayana, “Digital image forgery detection techniques: a survey,” ACCENTS Transactions on Information Security, vol. 2, no. 5, pp. 22–31, 2016, doi: 10.19101/TIS.2017.25003.
  7. Chauhan, D. Kasat, S. Jain, & V. Thakare, “Survey on keypoint based copy-move forgery detection methods on image,” Procedia Computer Science, vol. 85, pp. 206–212, 2016, doi: 10.1016/j.procs.2016.05.213.
  8. Agarwal, D. Khudaniya, A. Gupta, & K. Grover, “Image forgery detection and deep learning techniques: a review,” In 4th International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 1096–1100, IEEE, 2020, doi: 10.1109/ICICCS48265.2020.9121083.
  9. Z. Mehrjardi, A. M. Latif, M. S. Zarchi, & R. Sheikhpour, “A survey on deep learning-based image forgery detection,” Pattern Recognition, vol. 144, pp. 1-31, 2023, doi: 10.1016/j.patcog.2023.109778.
  10. Z. Mehrjardi, A. M. Latif, M. S. Zarchi, ” Copy-Move Forgery Detection and Localization Using Deep Learning,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 37, no. 9, pp. 1-21, 2023, doi: 10.1142/S0218001423520122.
  11. Z. Mehrjardi, A. M. Latif, M. S. Zarchi, “An Optimal Hybrid Method to Detect Copy-move Forgery,” Journal of AI and Data Mining, vol. 11, no. 3, pp. 429-442, 2023, doi: 10.22044/jadm.2023.13166.2453.
  12. Bravo-Solorio, & A. K. Nandi, “Automated detection and localization of duplicated regions affected by reflection, rotation and scaling in image forensics,” Signal Processing, vol. 91, no. 8, pp. 1759–1770, 2011, doi: 10.1016/j.sigpro.2011.01.022.
  13. Diallo, T. Urruty, P. Bourdon, & C. Fernandez-Maloigne, “Robust forgery detection for compressed images using CNN supervision,” Forensic Science International: Reports, vol. 2, pp. 100-112, 2020, doi: 10.1016/j.fsir.2020.100112.
  14. Thakur, & R. Rohilla, “Recent advances in digital image manipulation detection techniques: A brief review,” Forensic Science International, 2020, doi: 10.1016/j.forsciint.2020.110311.
  15. Mohamadian, & A. A. Pouyan, “Detection of duplication forgery in digital images in uniform and non-uniform regions,” 15th International Conference on Computer Modelling and Simulation, pp. 455–460, IEEE, 2013, doi: 10.1109/UKSim.2013.94.
  16. Zhang, Z. Liang, G. Yang, Q. Li, L. Li, & X. Sun, “A robust forgery detection algorithm for object removal by exemplar-based image inpainting,” Multimedia Tools and Applications, vol. 77, no. 10, pp. 11823–11842, 2018, doi: 10.1007/s11042-017-4829-0.
  17. Yang, H. Li, F. Lin, B. Jiang, & H. Zhao, “Constrained R-CNN: a general image manipulation detection model,” In IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, IEEE, 2020, doi: 10.1109/ICME46284.2020.9102825.
  18. Mahmood, T. Nawaz, R. Ashraf, M. Shah, Z. Khan, A. Irtaza, & Z. Mehmood, “A survey on block-based copy-move image forgery detection techniques,” In International Conference on Emerging Technologies (ICET), pp. 1–6, IEEE, 2015, doi: 10.1109/ICET.2015.7389169.
  19. D. Ferreira, C. B. Ferreira, G. da Cruz Júnior, & F. Soares, “A review of digital image forensics,” Computers & Electrical Engineering, vol. 85, 2020, doi: 10.1016/j.compeleceng.2020.106685.
  20. M. Al-Qureshi, & B. E. Khoo, “Evaluation of copy-move forgery detection: datasets and evaluation metrics,” Multimedia Tools and Applications, vol. 77, no. 24, pp. 31807–31833, 2018, doi: 10.1007/s11042-018-6201-4.
  21. Sharma, & U. Ghanekar, “A hybrid technique to discriminate natural images, computer generated graphics images, spliced, copy move tampered images and authentic images by using features and ELM classifier,” Optik, pp. 470–483, 2018, doi: 10.1016/j.ijleo.2018.07.021.
  22. Hoveida, & A. Shahbahrami, “Evaluating the effectiveness of block-based copy-move forgery detection,” Promotional scientific journal of soft computing, vol. 7, no. 1, pp. 62-79, 2017. (in Persian), doi: 10.22052/7.1.62.
  23. J. Sreelakshmy, & B. C. Kovoor, “Hybrid method for copy-move forgery detection in digital images,” In Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering (ISMAC-CVB), pp. 119–127, 2018, doi: 10.1007/978-3-030-00665-5_13.
  24. Mahmood, Z. Mehmood, M. Shah, & T. Saba, “A robust technique for copy-move forgery detection and localization in digital images via stationary wavelet and discrete cosine transform,” Journal of Visual Communication and Image Representation, vol. 53, pp. 202–214, 2018, doi: 10.1016/j.jvcir.2018.03.015.
  25. Koshy, & S. PraylaShyry, “Copy-move forgery detection and performance analysis of feature detectors,” In International Conference on Communication and Signal Processing (ICCSP), IEEE, pp. 41-45, 2020, doi: 10.1109/ICCSP48568.2020.9182066.
  26. Ouyang, Y. Liu, & M. Liao, “Copy-move forgery detection based on deep learning,” In 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), IEEE, pp. 1–5, 2017, doi: 10.1109/CISP-BMEI.2017.8301940.
  27. A. Elaskily, H. A. Elnemr, A. Sedik, M. M. Dessouky, G. M. El Banby, O. A. Elshakankiry, & F. E. Abd El-Samie, “A novel deep learning framework for copy-move forgery detection in images,” Multimedia Tools and Applications, vol. 79, no. 27, pp. 19167–19192, 2020, doi: 10.1007/s11042-020-08751-7.
  28. Agarwal, & O. P. Verma, “An efficient copy-move forgery detection using deep learning feature extraction and matching algorithm,” Multimedia Tools and Applications, vol. 79, no. 11, pp. 7355–7376, 2020, doi: 10.1007/s11042-019-08495-z.
  29. Rodríguez-Ortega, D. M. Ballesteros, & D. Renza, “Copy-move forgery detection (CMFD) using deep learning for image and video forensics,” Journal of Imaging, vol. 7, no. 3, 2021, doi: 10.3390/jimaging7030059.
  30. Goel, S. Kaur, & R. Bala, “Dual branch convolutional neural network for copy-move forgery detection,” IET Image Processing, vol. 15, no. 3, pp. 656–665, 2021, doi: 10.1049/ipr2.12051.
  31. N. Abbas, M. S. Ansari, M. N. Asghar, N. Kanwal, T. O'Neill, & B. Lee, “Lightweight deep learning model for detection of copy-move image forgery with post-processed attacks,” In IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 125-130, IEEE, 2021, doi: 10.1109/SAMI50585.2021.9378690.
  32. Doegar, M. Dutta, & K. Gaurav, “CNN-based image forgery detection using pre-trained AlexNet model,” International Journal of Computational Intelligence and IoT, 2019, [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3355402.
  33. T. H. Majumder, & A. A. Islam, “A tale of a deep learning approach to image forgery detection,” In 5th International Conference on Networking, Systems and Security (NSysS), pp. 1–9, IEEE, 2018, doi: 10.1109/NSysS.2018.8631389.
  34. M. Al_Azrak, A. Sedik, M. I. Dessowky, G. M. El Banby, A. A. Khalaf, A. S. Elkorany, & F. E. Abd. El-Samie, “An efficient method for image forgery detection based on trigonometric transforms and deep learning,” Multimedia Tools and Applications, vol. 79, no. 25, pp. 18221-18243, 2020, doi: 10.1007/s11042-019-08162-3.
  35. Doegar, S. Hiriyannaiah, S. G. Matt, S. K. Gopaliyengar, & M. Dutta, “Image forgery detection based on the fusion of lightweight deep learning models,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 29, no. 4, pp. 1978–1993, 2021, doi: 10.3906/elk-2005-37.
  36. Tralic, I. Zupancic, S. Grgic, & M. Grgic, “CoMoFoD -new database for copy-move forgery detection,” in Proceedings of 55th International Symposium ELMAR, pp. 49–54, IEEE, 2013.
  37. Amerini, L. Ballan, R. Caldelli, A. Del Bimbo, & G. Serra, “A sift-based forensic method for copy-move attack detection and transformation recovery,” IEEE transactions on information forensics and security, vol. 6, no. 3, pp. 1099–1110, 2011, doi: 10.1109/TIFS.2011.2129512.
  38. Tammina, "Transfer learning using VGG-16 with deep convolutional neural network for classifying images," International Journal of Scientific and Research (IJSRP), vol. 9, no. 10, 2019, doi: 10.29322/IJSRP.9.10.2019.p9420.
  39. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, & H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017, doi: 10.48550/arXiv.1704.04861.
  40. Tan, & Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” In International conference on machine learning, pp. 6105-6114, 2019, [Online]. Available:https://proceedings.mlr.press/v97/tan19a/tan19a.pdfChollet, “Deep learning with Python,” Simon and Schuster, 2021, doi: : 10.31211/interacoes.n42.2022.r1.
  41. A. Ghiahban, M. H. Shojaeifard, & A. Amirkhani, “Detection of Slippery Road Conditions using the Road CCTV Images based on the Convolutional Neural Networks and Transfer Learning,” Scientific Journal of Electronic and Cyber Defense, vol. 10, no. 2, pp. 103-114, 2022. (in Persian), dor: 20.1001.1.23224347.1401.10.2.9.5.
  42. Y. Hsiao, Y. C. Chang, H. H. Chou, & C. T. Chiu, “Filter-based deep-compression with global average pooling for convolutional networks,” Journal of Systems Architecture. Vol. 95, pp. 9–18, 2019, doi: 10.1016/j.sysarc.2019.02.008.
  43. Mahmood, “Copy move forgery detection technique for forensic analysis in digital images,” Mathematical Problems in Engineering. 2016, doi: 10.1155/2016/8713202.
  44. F. Hashmi, A. R. Hambarde & A. G. Keskar, “Copy move forgery detection using DWT and SIFT features,” In 13th International conference on intelligent systems design and applications, pp. 188–193, IEEE, 2013, doi: 10.1109/ISDA.2013.6920733.

Files

History

  • Receive Date: 27 August 2023
  • Revise Date: 25 November 2023
  • Accept Date: 16 December 2023
  • Publish Date: 18 January 2024

Share

How to cite

Statistics