ATSMF: Automated Tiered Storage with Fast Memory and Slow Flash Storage to Improve Response Time with Concentrated Input-Output (IO) Workloads
- 1 December 2018
- journal article
- research article
- Published by Institute of Electronics, Information and Communications Engineers (IEICE) in IEICE Transactions on Information and Systems
- Vol. E101.D (12), 2889-2901
- https://doi.org/10.1587/transinf.2018pap0005
Abstract
The response times of solid state drives (SSDs) have decreased dramatically due to the growing use of non-volatile memory express (NVMe) devices. Such devices have response times of less than 100 micro seconds on average. The response times of all-flash-array systems have also decreased dramatically through the use of NVMe SSDs. However, there are applications, particularly virtual desktop infrastructure and in-memory database systems, that require storage systems with even shorter response times. Their workloads tend to contain many input-output (IO) concentrations, which are aggregations of IO accesses. They target narrow regions of the storage volume and can continue for up to an hour. These narrow regions occupy a few percent of the logical unit number capacity, are the target of most IO accesses, and appear at unpredictable logical block addresses. To drastically reduce the response times for such workloads, we developed an automated tiered storage system called “automated tiered storage with fast memory and slow flash storage” (ATSMF) in which the data in targeted regions are migrated between storage devices depending on the predicted remaining duration of the concentration. The assumed environment is a server with non-volatile memory and directly attached SSDs, with the user applications executed on the server as this reduces the average response time. Our system predicts the effect of migration by using the previously monitored values of the increase in response time during migration and the change in response time after migration. These values are consistent for each type of workload if the system is built using both non-volatile memory and SSDs. In particular, the system predicts the remaining duration of an IO concentration, calculates the expected response-time increase during migration and the expected response-time decrease after migration, and migrates the data in the targeted regions if the sum of response-time decrease after migration exceeds the sum of response-time increase during migration. Experimental results indicate that ATSMF is at least 20% faster than flash storage only and that its memory access ratio is more than 50%.Keywords
This publication has 11 references indexed in Scilit:
- Automated Tiered Storage System Consisting of Memory and Flash Storage to Improve Response Time with Input-Output (IO) Concentration WorkloadsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Feasibility Study for Building Hybrid Storage System Consisting of Non-volatile DIMM and SSDPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Analysis of Storage Workloads of Input-Output Access Locality and Designing of Hybrid Storage SystemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Algorithm-Directed Data Placement in Explicitly Managed Non-Volatile MemoryPublished by Association for Computing Machinery (ACM) ,2016
- A high performance file system for non-volatile main memoryPublished by Association for Computing Machinery (ACM) ,2016
- Data tiering in heterogeneous memory systemsPublished by Association for Computing Machinery (ACM) ,2016
- Multi-cache: Dynamic, Efficient Partitioning for Multi-tier Caches in Consolidated VM EnvironmentsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- On-the-Fly Automated Storage Tiering with Caching and both Proactive and Observational MigrationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Improving flash-based disk cache with Lazy Adaptive ReplacementPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Write off-loadingACM Transactions on Storage, 2008