Dynamic Memory Partitioning for Cloud Caches with Heterogeneous Backends

Abstract

Software caches, implemented with in-memory key-value stores, are important components of cloud architectures. In a common scenario, one server may serve requests from several applications with different workloads, each supported by a different backend (database or storage system); these applications compete for an allocation of the total memory. We present a model for dynamic memory partitioning for cloud caches with heterogeneous backends. Our work differs from recent work for cloud caches in that we consider: (1) the effect of having backends with different performance profiles, and (2) the cost of re-partitioning the memory. We discuss implementation issues that must be addressed, including the need for on-line and lightweight mechanisms for estimating the miss rate curves (MRCs) and ways to solve the non-convex optimization problem; specifically, we propose a probabilistic adaptive search algorithm that can be used for discontinuous, non-differentiable, or non-convex MRCs.