Autonomous Power Management With Double-QReinforcement Learning Method

Abstract

Energy efficiency and autonomous power management are extremely important for mobile edge computing. Reducing energy consumption of a number of applications running concurrently in mobile devices while maintaining performance poses a challenge to energy optimization due to the limited capacity of the embedded battery. To extend battery life and offer a long-lasting working energy, dynamic voltage and frequency scaling (DVFS) has been widely used in mobile devices for energy consumption minimization. However, most conventional DVFS techniques scale operating frequency based on static policies, and thus they are difficult to be adapted to systems of varied conditions. In order to improve adaptivity, we proposed a Double- Q power management approach to scale operating frequency based on learning. The Double-Q method stores two Q-tables and two corresponding update functions. In each decision point, either of Q-tables is randomly chosen and updated, while the other is used for the measurement. This mechanism reduces the overestimation in Q-values, consequently enhancing the accurateness of frequency predictions. To evaluate the effectiveness of our proposed approach, a Double-Q governor is implemented in the Linux kernel. Our approach is computationally light and experimental results indicate that it achieves at least 5%-18% total energy saving compared to ondemand and conservative governors as well as Q learning-based method.