Classification of nodes in citation graphs using graph neural networks

Document Type : Original Article

Authors

1 Master's student, Imam Hossein University (AS), Tehran, Iran

2 PhD student, Imam Hossein University (AS), Tehran, Iran

3 Assistant Professor, Imam Hossein University (AS), Tehran, Iran

Abstract

Graphs are data that describe complex relationships between different things in the real world, such as the Internet, social network, bibliographic network, and so on. One of the things that many people deal with today is online social networks. The graph display of online social networks such as Twitter, WeChat and Facebook is not possible today with less than billions of nodes, and for this reason, the study of large-scale network data has become a necessity for researchers. Regarding social networks, online users often have limited information; But for social media service providers, user node information such as interest, beliefs, or other characteristics are very important to customize their services for users in many applications such as recommendations and personalized search, making it a challenge for service providers. An effective way to deal with this challenge is to infer missing user information using pervasive network structures in social media. One of the most important inferences in data mining and network analysis is node classification, which aims to infer the missing labels of nodes based on labeled nodes and network structure. In this research, we have performed the task of node classification on the PubMedDiabetes, CiteSeer and Cora citation network datasets using GraphSAGE, GCN and GAT neural networks and we have generally concluded that the GraphSAGE neural network on the network datasets The cited reference works well for the node classification task.

Keywords


Smiley face

[1] 
Weibo Liu, Zidong Wang, Xiaohui Liu, Nianyin Zeng, Yurong Liu, Fuad E. Alsaadi, "A survey of deep neural network architectures and their applications," Neurocomputing, vol. 234, pp. 11 - 26, 2017. 
[2] 
Qingchen Zhang, Laurence T. Yang, Zhikui Chen, Peng Li , "A survey on deep learning for big data," Information Fusion, vol. 42, pp. 146-157, 2018. 
[3] 
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," in 26th Conference on Neural Information Processing Systems, 2012. 
[4] 
Weitao Wan, Yuanyi Zhong, Tianpeng Li, Jiansheng Chen, "Rethinking Feature Distribution for Loss Functions in Image Classification," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. 
[5] 
Jonathan Long, Evan Shelhamer, Trevor Darrell, "Fully convolutional networks for semantic segmentation," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 2015. 
[6] 
Maria Papadomanolaki, Maria Vakalopoulou, Konstantinos Karantzalos, "A Novel Object-Based Deep Learning Framework for Semantic Segmentation of Very High-Resolution Remote Sensing Data: Comparison with Convolutional and Fully Convolutional Networks," Remote Sensing, vol. 11, no. 6, 2019. 
[7] 
Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, Tat-Seng Chua, "SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning," in 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017. 
[8] 
Jyoti Aneja, Aditya Deshpande, Alexander Schwing, "Convolutional Image Captioning," in 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018. 
[9] 
Sepp Hochreiter, Jürgen Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997. 
[10] 
Duyu Tang, Bing Qin, Ting Liu, "Document Modeling with Gated Recurrent Neural Network for Sentiment Classification," in 2015 Conference on Empirical Methods in Natural Language Processing, 2015. 
[11] 
Yukun Ma, Haiyun Peng, Erik Cambria, "Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM," in 32 AAAI Conference on Artificial Intelligence, New Orleans, 2018. 
[12] 
Shujie Liu, Nan Yang, Mu Li, Ming Zhou, "A Recursive Recurrent Neural Network for Statistical Machine Translation," in 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, 2014. 
[13] 
Jinsong Su. Shan Wu, Deyi Xiong, Yaojie Lu, Xianpei Han, Biao Zhang, "Variational Recurrent Neural Machine Translation," in 32 AAAI Conference on Artificial Intelligence, New Orleans, 2018. 
[14] 
Caiming Xiong, Stephen Merity, Richard Socher, "Dynamic Memory Networks for Visual and Textual Question Answering," in 33nd International Conference on Machine Learning, 2016. 
[15] 
Yankai Lin, Haozhe Ji, Zhiyuan Liu, Maosong Sun, "Denoising Distantly Supervised Open-Domain Question Answering," in 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, 2018. 
[16] 
Edward T Bullmore, Olaf Sporns, "Complex brain networks: Graph theoretical analysis of structural and functional systems," Nature Reviews Neuroscience, vol. 10, no. 3, 2009. 
[17] 
Meysam Mirzaei, Aminollah Mahabadi, "Hybrid Anomaly Detection Method Using Community Detection in Graph and Feature Selection," Journal of Electronical & Cyber Defence, vol. 8, no. 1, pp. 17-24, 2020. (In Persian)
[18] 
James Atwood, Don Towsley, "Diffusion-Convolutional Neural Networks," in 30th Conference on Neural Information Processing Systems, 2016. 
[19] 
Qimai Li, Zhichao Han, Xiao-ming Wu, "Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning," in 32nd AAAI Conference on Artificial Intelligence, New Orleans, 2018. 
[20] 
Muhan Zhang, Yixin Chen, "Link Prediction Based on Graph Neural Networks," in 32nd Conference on Neural Information Processing Systems, 2018. 
[21] 
Xiaojun Xu, Chang Liu, Qian Feng, Heng Yin, Le Song, Dawn Song, "Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detection," in 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017. 
[22] 
Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov, "Learning Convolutional Neural Networks for Graphs," in 33rd International Conference on Machine Learning, 2016. 
[23] 
Muhan Zhang, Zhicheng Cui, Marion Neumann, Yixin Chen, "An End-to-End Deep Learning Architecture for Graph Classification," in 32nd AAAI Conference on Artificial Intelligence, 2018. 
[24] 
Przemyslaw Kazienko , Tomasz Kajdanowicz, "Label-dependent node classification in the network," Neurocomputing, vol. 75, no. 1, pp. 199 - 209, 2012. 
[25] 
HongYun Cai, Vincent W. Zheng, Kevin Chen-Chuan Chang, "A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications," IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 9, pp. 1616 - 1637, 2018. 
[26] 
Qimai Li, Zhichao Han, Xiao-Ming Wu, "Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning," in 32 AAAI Conference on Artificial Intelligence, Menlo Park, 2018. 
[27] 
Guohao Li, Matthias Müller, Guocheng Qian, Itzel C. Delgadillo, Abdulellah Abualshour, Ali Thabet, Bernard Ghanem, "DeepGCNs: Making GCNs Go as Deep as CNNs," in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 2019. 
[28] 
Bryan Perozzi, Rami Al-Rfou, Steven Skiena, "DeepWalk: Online Learning of Social Representations," in 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2014. 
[29] 
Aditya Grover, Jure Leskovec, "node2vec: Scalable Feature Learning for Networks," in 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016. 
[30] 
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, Qiaozhu Mei, "LINE: Large-scale Information Network Embedding," in 24th International Conference onWorld Wide Web, Montreal, 2015. 
[31] 
Thomas N. Kipf, Max Welling, "Semi-Supervised Classification with Graph Convolutional Networks," in 4th International Conference on Learning Representations, 2016. 
[32] 
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio, "Graph Attention Networks," in 5th International Conference on Learning Representations, 2017. 
[33] 
William L. Hamilton, Rex Ying, Jure Leskovec, "Inductive Representation Learning on Large Graphs," in 31st Conference on Neural Information Processing Systems, 2017. 
[34] 
Chao Li, Li Wang, Shiwen Sun, Chengyi Xia, "Identification of influential spreaders based on classified neighbors in real-world complex networks," Applied Mathematics and Computation, vol. 320, pp. 512-523, 2017. 
[35] 
Etienne Gael Tajeuna, Mohamed Bouguessa, Shengrui Wang, "Modeling and Predicting Community Structure Changes in Time-Evolving Social Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 6, pp. 1166 - 1180, 2019. 
[36] Yizhou Sun, Yintao Yu, Jiawei Han, "Ranking-based Clustering of Heterogeneous Information Networks with star Network Schema," in 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2009. 

[37] 
Vincenzo Nicosia, Vito Latora, "Measuring and modelling correlations in multiplex networks," PHYSICAL REVIEW E, 2015. 
[38] 
Zhishuang Wang, Quantong Guo, Shiwen Sun, Chengyi Xia, "The impact of awareness diffusion on SIR-like epidemics in multiplex networks," Applied Mathematics and Computation, vol. 349, pp. 134-147, 2019. 
[39] Shunxin Xiao, Shiping Wang, Yuanfei Dai, Wenzhong Guo, "Graph neural networks in node classification: survey and evaluation," Machine Vision and Applications, vol. 33, no. 1, 2022. 

[40] Jiawei Zhang, Haopeng Zhang, Congying Xia, Li Sun, "Graph-Bert: Only Attention is Needed for Learning Graph Representations," 2020. 

  • Receive Date: 26 December 2023
  • Revise Date: 13 April 2024
  • Accept Date: 03 May 2024
  • Publish Date: 21 May 2024