DQELR: An Adaptive Deep Q-Network-Based Energy- and Latency-Aware Routing Protocol Design for Underwater Acoustic Sensor Networks
Open Access
- 9 January 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 9091-9104
- https://doi.org/10.1109/access.2019.2891590
Abstract
Underwater acoustic sensor networks (UASNs) have become a popular research topic, with research challenges focused on underwater communication techniques. By incorporating long end-to-end latency, high energy consumption and dynamic network topology in UASNs, many intelligent routing protocols have been proposed to solve the problem. However, shortcomings still exist, and comprehensive routing protocols are urgently needed. In this paper, we propose an adaptive Deep Q-Network-based energy- and latency-aware routing protocol (DQELR) to prolong network lifetimes in UASNs. In the DQELR, a Deep Q-Network algorithm with both off-policy and on-policy methods is adopted to make globally optimal routing decisions. Based on both the energy and depth states of nodes at different communication stages, nodes with the maximum Q-value can be selected as forwarders adaptively considering both energy and latency. A hybrid of the broadcast and unicast communication mechanisms is also designed to reduce network overhead. In addition, network topology changes can be addressed through an on-policy method that makes a new routing decision when the current route becomes corrupted. With less energy consumption and strict latency limitations, the DQELR can prolong network lifetimes in UASNs. Simulation results show that the DQELR can achieve a superior network lifetime with better latency and energy efficiency performances relative to other general schemes applied in UASNs.Funding Information
- National Natural Science Foundation of China (61701335, 61862020, 61861014, 61571323, 61571318)
- Science and Technology on Underwater Information and Control Laboratory (614221801050517)
- Natural Science Foundation of Tianjin City (17JCQNJC01300)
- Key Research and Development Plan of Hainan (ZDYF2018006)
This publication has 25 references indexed in Scilit:
- An adaptive routing protocol in underwater sparse acoustic sensor networksAd Hoc Networks, 2015
- Node Deployment Algorithm Based on Connected Tree for Underwater Sensor NetworksSensors, 2015
- Distance based Reliable and Energy Efficient (DREE) Routing Protocol for Underwater Acoustic Sensor NetworksJournal of Networks, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Cluster based routing algorithm with evenly load distribution for large scale networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- DSP based receiver implementation for OFDM acoustic modemsPhysical Communication, 2012
- Adaptive Power Controlled Routing for Underwater Sensor NetworksLecture Notes in Computer Science, 2012
- Improving the Robustness of Location-Based Routing for Underwater Sensor NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2007
- The WHOI Micro-Modem: An Acoustic Communications and Navigation System for Multiple PlatformsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- On the lifetime of wireless sensor networksIEEE Communications Letters, 2005