Nimble Page Management for Tiered Memory Systems
- 4 April 2019
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
Software-controlled heterogeneous memory systems have the potential to increase the performance and cost efficiency of computing systems. However they can only deliver on this promise if supported by efficient page management policies and mechanisms within the operating system (OS). Current OS implementations do not support efficient tiering of data between heterogeneous memories. Instead, they rely on expensive offlining of memory or swapping data to disk as a means of profiling and migrating hot or cold data between memory nodes. They also leave numerous optimizations on the table; for example, multi-threaded hardware is not leveraged to maximize page migration throughput, resulting in up to 95% under-utilization of available memory bandwidth. To remedy these shortcomings, we propose and implement a general purpose OS-integrated multi-level memory management system that reuses current OS page tracking structures to tier pages directly between memories with no additional monitoring overhead. We augment this system with four additional optimizations: native support for transparent huge page migration, multi-threaded migration of a page, concurrent migration of multiple pages, and symmetric exchange of pages. Combined, these optimizations dramatically reduce kernel software overheads and improve raw page migration throughput over 15×. Implemented in Linux and evaluated on x86, Power, and ARM64 systems, our OS support for heterogeneous memories improves application performance 40% over baseline Linux for a suite of real-world memory-intensive workloads utilizing a multi-level disaggregated memory system.Keywords
Funding Information
- National Science Foundation (1253700, 1337147, 1319755)
This publication has 48 references indexed in Scilit:
- Redundant memory mappings for fast access to large memoriesPublished by Association for Computing Machinery (ACM) ,2015
- HeteroVisorPublished by Association for Computing Machinery (ACM) ,2015
- Page Placement Strategies for GPUs within Heterogeneous Memory SystemsPublished by Association for Computing Machinery (ACM) ,2015
- BadgerTrapACM SIGARCH Computer Architecture News, 2014
- Concurrent page migration for mobile systems with OS-managed hybrid memoryPublished by Association for Computing Machinery (ACM) ,2014
- Architectural support for address translation on GPUsPublished by Association for Computing Machinery (ACM) ,2014
- NUMA (Non-Uniform Memory Access): An OverviewQueue, 2013
- Hardware monitors for dynamic page migrationJournal of Parallel and Distributed Computing, 2008
- Concurrent support of multiple page sizes on a skewed associative TLBInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2004
- Reflections on the memory wallPublished by Association for Computing Machinery (ACM) ,2004