Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing
Top Cited Papers
- 20 June 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 68 (8), 8050-8062
- https://doi.org/10.1109/tvt.2019.2924015
Abstract
Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize long-term offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Lastly, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies.Keywords
Funding Information
- National Key Research and Development Plan (2018YFB1003803)
- National Natural Science Foundation of China (61802450, 61722214)
- Natural Science Foundation of Guangdong Province (2018A030313005)
- Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2017ZT07X355)
This publication has 41 references indexed in Scilit:
- User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning ApproachIEEE Transactions on Wireless Communications, 2017
- AC-POCA: Anticoordination Game Based Partially Overlapping Channels Assignment in Combined UAV and D2D-Based NetworksIEEE Transactions on Vehicular Technology, 2017
- Joint Computation Offloading and Interference Management in Wireless Cellular Networks with Mobile Edge ComputingIEEE Transactions on Vehicular Technology, 2017
- Device-to-Device Communication for Mobile Multimedia in Emerging 5G NetworksACM Transactions on Multimedia Computing, Communications, and Applications, 2016
- Smart Home Communication Technologies and Applications: Wireless Protocol Assessment for Home Area Network ResourcesEnergies, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Device-to-Device Communication in LTE-Advanced Networks: A SurveyIEEE Communications Surveys & Tutorials, 2014
- Relay-by-smartphone: realizing multihop device-to-device communicationsIEEE Communications Magazine, 2014
- Adaptive probabilities of crossover and mutation in genetic algorithmsIEEE Transactions on Systems, Man, and Cybernetics, 1994
- Q-learningMachine Learning, 1992