Hierarchical dynamic power management using model-free reinforcement learning
- 1 March 2013
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in International Symposium on Quality Electronic Design (ISQED)
Abstract
Model-free reinforcement learning (RL) has become a promising technique for designing a robust dynamic power management (DPM) framework that can cope with variations and uncertainties that emanate from hardware and application characteristics. Moreover, the potentially significant benefit of performing application-level scheduling as part of the system-level power management should be harnessed. This paper presents an architecture for hierarchical DPM in an embedded system composed of a processor chip and connected I/O devices (which are called system components.) The goal is to facilitate saving in the system component power consumption, which tends to dominate the total power consumption. The proposed (online) adaptive DPM technique consists of two layers: an RL-based component-level local power manager (LPM) and a system-level global power manager (GPM). The LPM performs component power and latency optimization. It employs temporal difference learning on semi-Markov decision process (SMDP) for model-free RL, and it is specifically optimized for an environment in which multiple (heterogeneous) types of applications can run in the embedded system. The GPM interacts with the CPU scheduler to perform effective application-level scheduling, thereby, enabling the LPM to do even more component power optimizations. In this hierarchical DPM framework, power and latency tradeoffs of each type of application can be precisely controlled based on a user-defined parameter. Experiments show that the amount of average power saving is up to 31.1% compared to existing approaches.Keywords
This publication has 11 references indexed in Scilit:
- Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classificationPublished by Association for Computing Machinery (ACM) ,2011
- Dynamic Power Management Using Machine Learning2006 IEEE/ACM International Conference on Computer Aided Design, 2006
- Hierarchical power management with application to schedulingPublished by Association for Computing Machinery (ACM) ,2005
- Managing power consumption in networks on chipsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Dynamic power management based on continuous-time Markov decision processesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Event-driven power managementIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2001
- A survey of design techniques for system-level dynamic power managementIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2000
- A predictive system shutdown method for energy saving of event-driven computationACM Transactions on Design Automation of Electronic Systems, 2000
- Policy optimization for dynamic power managementIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 1999
- Predictive system shutdown and other architectural techniques for energy efficient programmable computationIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 1996