Classification of High Dimensional Imbalanced Dataset via Game Theory-based Generative Adversarial Networks

Document Type : Original Article

Authors

1 Faculty member of Payam Nour

2 Faculty member of IUST

3 Faculty member of Karaj branch fo Azad university

Abstract

Game theory uses mathematical models to analyze the methods of cooperation or competition of intelligent beings. Game theory attempts to model the mathematical behavior of strategic interaction among rational decision-makers. The ultimate goal of this knowledge is to find the optimal strategy for the players. One of the newest ideas in the application of Game theory in the field of artificial intelligence and machine learning is Generative Adversarial Networks. GANs consist of two parts, use Game theory and compete with each other, making it possible for unsupervised or semi-supervised learning. In addition to generating data, these networks are also used to identify malicious software and software security, machine translation, and natural language processing, and to build a three-dimensional model of an image. However, GANs have a very long training time due to the high number of epochs and input parameters. In this paper, in order to solve the problem of long training time of these networks in the classification of imbalanced high-dimensional datasets, a solution is presented that first, GAN-based oversampling on minority classes. Then in order to improve the efficiency of the designed GAN, the mentioned network is parallelized and ensemble classification is done. The different scenarios performed on the classification of diabetic retinopathy dataset by the proposed method.The results showed the classification accuracy of 87%, the training time is reduced by 74%, which shows higher accuracy than the latest scientific advances.

Keywords

References

[1]             M. R. K. H. Saberi and M. R. Hasani Ahangar, “Providing an Agent-Based Architecture for Semantic Mining From Large-Scale Data in Distributed Environments,” Journal of Electronical & Cyber Defence, vol. 8, no. 3, 2020.##
[2]             S. Krig, “Feature learning and deep learning architecture survey,” in Computer Vision Metrics: Springer, pp. 375-514. 2016.##
[3]                 I. Goodfellow, Y. Bengio, and A. Courville, Deep learning, MIT press, 2016.##
[4]             S. P. M. Zakeri Nasrabadi1, “Automatic Test Data Generation in File Format Fuzzers,” Journal of Electronical & Cyber Defence, vol. 8, no. 29, 2020.##
[5]             D. C. Ciresan, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber, “Flexible, high performance convolutional neural networks for image classification,” in Twenty-Second International Joint Conference on Artificial Intelligence, 2011.##
[6]             C. C. Chatterjee, “Basics of the Classic CNN,” https://towardsdatascience.com/basics-of-the-classic-cnn-a3dce1225add (accessed may/11/2019).##
[7]                “Convolutional Neural Networks (CNNs / ConvNets),” https://cs231n.github.io/convolutional-networks/ (accessed.##
[8]             Y. Sun, A. K. Wong, and M. S. Kamel, “Classification of imbalanced data: A review,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 23, no. 04, pp. 687-719,2009.##
[9]             M. Buda, A. Maki, and M. A. Mazurowski, “A systematic study of the class imbalance problem in convolutional neural networks,” Neural Networks, vol. 106, pp. 249-259, 2018.##
[10]          J. v. Neumann, “Zur theorie der gesellschaftsspiele,” Mathematische annalen, vol. 100, no. 1, pp. 295-320, 1928.##
[11]          A. W. Tucker and R. D. Luce, Contributions to the Theory of Games (no. 40), Princeton University Press, 1959.##
[12]          A. A. M. Forooghy and M. Bagheri, “A Decision-Making Model in a Cyber Conflicts Acted Upon Vulnerability, Based on Game Theoretic Analysis,” Journal of Electronical & Cyber Defence, vol. 6, no. 22, 2018.##
[13]               I. Goodfellow et al., “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672-2680, 2014.##
[14]            E. L. Denton, S. Chintala, and R. Fergus, “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, pp. 1486-1494, 2015.##
[15]          I. Goodfellow, “NIPS 2016 tutorial: Generative adversarial networks,” arXiv preprint arXiv:1701.00160, 2016. [Online]. Available: http://arxiv.org/abs/1701.00160.##
[16]          A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.##
[17]          M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014.##
[18]          J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” pecial Lecture on IE, vol. 2, no. 1, 2015.##
[19]           S. Nowozin, B. Cseke, and R. Tomioka, “f-gan: Training generative neural samplers using variational divergence minimization,” in Advances in neural information processing systems, pp. 271-279, 2016.##
[20]          A. Kadurin, S. Nikolenko, K. Khrabrov, A. Aliper, and A. Zhavoronkov, “druGAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico,” Molecular pharmaceutics, vol. 14, no. 9, pp. 3098-3104, 2017.##
[21]          T. G. Dietterich, “Ensemble learning,” The handbook of brain theory and neural networks, vol. 2, pp. 110-125, 2002.##
[22]          L. Torrey and J. Shavlik, “Transfer learning,” in Handbook of research on machine learning applications and trends: algorithms, methods, and techniques: IGI global, pp.        242-264, 2010.##
[23]          F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size,” arXiv preprint arXiv:1602.07360, 2016.##
[24]           O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234-241, 2015.##
[25]           H. Qassim, A. Verma, and D. Feinzimer, “Compressed residual-VGG16 CNN model for big data places image recognition,” in 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), IEEE, pp. 169-175, 2018.##
[26]          F. C. e. al., “Keras” https://keras.io (accessed).##
[27]          M. Abadi et al., “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint arXiv:1603.04467, 2016.##
[28]          L. ZeBlemoyer. “Linear Regression Bias / Variance Tradeoff.” https://courses.cs.washington.edu/courses/cse546/ (accessed 2018).##
[29]          Y. Bengio, “Practical recommendations for gradient-based training of deep architectures,” in Neural networks: Tricks of the trade, Springer, pp. 437-478, 2012.##
[30]          J. Hermans, “On Scalable Deep Learning and Parallelizing Gradient Descent,” Master, Maastricht, 2017. [Online]. Available: http://cds.cern.ch/record/2276711.##
[31]          D. Masters and C. Luschi, “Revisiting Small Batch Training for Deep Neural Networks. arXiv 2018,” arXiv preprint arXiv:1804.07612.##
[32]          N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, “On large-batch training for deep learning: Generalization gap and sharp minima,” arXiv preprint arXiv:1609.04836, 2016. [Online]. Available: http://arxiv.org/abs/1609.04836.##
[33]          B. Graham, “Kaggle diabetic retinopathy detection competition report,” University of Warwick, 2015.##
[34]          M. Antony and S. Brüggemann, “Kaggle Diabetic Retinopathy Detection; Team o_O solution,” ed: Competition Report Github. url: https://github. com/sveitser/kaggle_diabetic …, 2015.##
[35]          S. Qummar et al., “A deep learning ensemble approach for diabetic retinopathy detection,” IEEE Access, vol. 7, pp. 150530-150539, 2019.##
           [36] H. Pratt, F. Coenen, D. M. Broadbent, S. P. Harding, and Y. Zheng, “Convolutional neural networks for diabetic retinopathy,” Procedia Computer Science, vol. 90, pp.  200-205, 2016.##
Electronic and Cyber Defense
Volume 9, Issue 2 - Serial Number 34
Serial No. 34, Summer Quarterly
June 2021
Pages 63-74

Files

History

  • Receive Date: 30 July 2020
  • Revise Date: 28 November 2020
  • Accept Date: 11 January 2021
  • Publish Date: 22 June 2021

Share

How to cite

Statistics