A Dueling Deep Recurrent Q -Network Framework for Dynamic Multichannel Access in Heterogeneous Wireless Networks
Open Access
- 1 October 2022
- journal article
- research article
- Published by Hindawi Limited in Wireless Communications and Mobile Computing
- Vol. 2022, 1-14
- https://doi.org/10.1155/2022/9446418
Abstract
This paper investigates a deep reinforcement learning algorithm based on dueling deep recurrent -network (Dueling DRQN) for dynamic multichannel access in heterogeneous wireless networks. Specifically, we consider the scenario that multiple heterogeneous users with different MAC protocols share multiple independent channels. The goal of the intelligent node is to learn a channel access strategy that achieves high throughput by making full use of the underutilized channels. Two key challenges for the intelligent node are (i) there is no prior knowledge of spectrum environment or the other nodes’ behaviors; (ii) the spectrum environment is partially observable, and the spectrum states have complex temporal dynamics. In order to overcome the aforementioned challenges, we first embed the long short-term memory layer (LSTM) into the deep -network (DQN) to aggregate historical observations and capture the underlying temporal feature in the heterogeneous networks. And second, we employ the dueling architecture to overcome the observability problem of dynamic environment in neural networks. Simulation results show that our approach can learn the optimal access policy in various heterogeneous networks and outperforms the state-of-the-art policies.Keywords
Funding Information
- National Natural Science Foundation of China (62171449, 62001483, 61931020)
This publication has 34 references indexed in Scilit:
- Licensed Spectrum Sharing Schemes for Mobile Operators: A Survey and OutlookIEEE Communications Surveys & Tutorials, 2016
- Digital architecture to implement a piecewise-linear approximation for the hyperbolic tangent functionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Database-assisted dynamic spectrum access with QoS guarantees: A double-phase auction approachChina Communications, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Distributed Q-learning based dynamic spectrum access in high capacity density cognitive cellular systems using secondary LTE spectrum sharingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- On Optimality of Myopic Sensing Policy with Imperfect Sensing in Multi-Channel Opportunistic AccessIEEE Transactions on Communications, 2013
- Q-learning cell selection for femtocell networks: Single- and multi-user casePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel AccessIEEE Transactions on Information Theory, 2010
- Network reconfiguration of the shipboard power system based on logistic function particle swarm optimizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- Learning to Forget: Continual Prediction with LSTMNeural Computation, 2000