Reducing the Destructive Effect of Misbehaving Users in Cooperative Spectrum Sensing using Reinforcement Learning

Document Type : Original Article

Author

PhD student, Aras International Branch, Islamic Azad University, Tabriz, Iran

Abstract

The presence of misbehaving users in Cognitive Radio Networks (CRN) can disrupt the process of spectrum sensing and detecting the status of the Primary User (PU). In order to reduce the destructive effect of this group of users in CRNs, in this paper, a new mechanism based on reinforcement learning for cooperative spectrum sensing is presented. The proposed method is a cooperative spectrum sensing mechanism based on user weighting, according to which users receive a weight commensurate with how they behave in spectrum sensing. The reinforcement learning model used in the proposed method is a learning automata which, using reward and penalty processes, allocates more weight to users with normal behavior in sensing the spectrum and less to misbehaving users. In this way, the learning automata updates the users' weight vector based on the response received from the environment, after performing a sensing operation in each repetition. After repeating the sensing operation several times, the learner will be able to optimize the user's weight vector. In order to evaluate the proposed method, its performance in the simulation environment has been tested and the results have been compared with the existing method for cooperative spectrum sensing. The results show that using the proposed method in the presence of misbehaving users will significantly improve network performance.

Keywords

Smiley face

References

[1] ‏S. Khamayseh, & A. Halawani, “Cooperative Spectrum Sensing in Cognitive Radio Networks: A Survey on Machine Learning-based Methods,” Journal of Telecommunications and Information Technology, Vol. 82, pp. 1-18, 2020.
[2] P.T. Sivagurunathan, P.  Ramakrishnan, & N. Sathishkumar, “Recent Paradigms for Efficient Spectrum Sensing in Cognitive Radio Networks: Issues and Challenges,” Journal of Physics: Conference Series, Vol. 1717, pp. 12-57, 2021.
[3] F. Benedetto, & G. Giunta, “A theoretical analysis of asymptotical performance of cooperative spectrum sensing in the presence of malicious users,” IEEE Wireless Communications Letters, Vol. 7, pp. 380-383, 2017.
[4] M. Botvinick, S. Ritter, J. Wang, Z. Kurth-Nelson, C. Blundell, & D. Hassabis, “Reinforcement learning, fast and slow,” Trends in cognitive sciences, Vol. 23, pp. 408-422, 2019.
[5] S. Levine, A. Kumar, G. Tucker, & J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005. 01643, 2020.
[6] A. Sharifi, M. Sharifi, and J. Niya, “Secure cooperative spectrum sensing under primary user emulation attack in cognitive radio networks:  Attack-aware threshold selection approach,” International Journal of Electronics and Communications (AEÜ), Vol. 10, pp. 1-10, 2015.
[7] X. Liu, C. Sun, M. Zhou, C. Wu, B. Peng, & P. Li, “Reinforcement learning-based multislot double-threshold spectrum sensing with Bayesian fusion for industrial big spectrum data,” IEEE Transactions on Industrial Informatics, Vol. 17, pp. 3391-3400, 2020.
[8] Y. Zhang, P. Cai, C. Pan, & S. Zhang, “Multi-agent deep reinforcement learning-based cooperative spectrum sensing with upper confidence bound exploration,” IEEE Access, Vol. 7, pp. 118898-118906, 2019.
[9] A. Kumar, N. Gupta, R. Tapwal, & J. Singh, “Trust Aware Scheme based Malicious Nodes Detection under Cooperative Spectrum Sensing for Cognitive Radio 
Networks,” Adjunct Proceedings of the International Conference on Distributed Computing and Networking, Vol. 21, pp. 56-61, 2021.
[10] R. Wan, L. Ding, N. Xiong, W. Shu, & L. Yang, “Dynamic dual threshold cooperative spectrum sensing for cognitive radio under noise power uncertainty,” Human-centric Computing and Information Sciences, Vol. 9, pp. 1-21, 2019.
[11] W. Ning, X. Huang, K. Yang, F. Wu, & S. Leng, “Reinforcement learning enabled cooperative spectrum sensing in cognitive radio networks,” Journal of Communications and Networks, Vol. 22, pp. 12-22, 2020.
[12] M. Rajendran, & M. Duraisamy, “Distributed coalition formation game for enhancing cooperative spectrum sensing in cognitive radio ad hoc networks,” IET Networks, Vol. 9, pp. 12-22, 2020.
[13] E. Ghazizadeh, D. Abbasi‐moghadam, & H. Nezamabadi‐pour, “An enhanced two‐phase SVM algorithm for cooperative spectrum sensing in cognitive radio networks,” International Journal of Communication Systems, Vol. 32, pp. 38-56, 2019.

Files

History

  • Receive Date: 08 September 2021
  • Revise Date: 05 December 2021
  • Accept Date: 09 August 2022
  • Publish Date: 21 January 2023

Share

How to cite

Statistics