Using RDMA efficiently for key-value services
Open Access
- 17 August 2014
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM SIGCOMM Computer Communication Review
- Vol. 44 (4), 295-306
- https://doi.org/10.1145/2740070.2626299
Abstract
This paper describes the design and implementation of HERD, a key-value system designed to make the best use of an RDMA network. Unlike prior RDMA-based key-value systems, HERD focuses its design on reducing network round trips while using efficient RDMA primitives; the result is substantially lower latency, and throughput that saturates modern, commodity RDMA hardware. HERD has two unconventional decisions: First, it does not use RDMA reads, despite the allure of operations that bypass the remote CPU entirely. Second, it uses a mix of RDMA and messaging verbs, despite the conventional wisdom that the messaging primitives are slow. A HERD client writes its request into the server's memory; the server computes the reply. This design uses a single round trip for all requests and supports up to 26 million key-value operations per second with 5μs average latency. Notably, for small key-value items, our full system throughput is similar to native RDMA read throughput and is over 2X higher than recent RDMA-based key-value systems. We believe that HERD further serves as an effective template for the construction of RDMA-based datacenter services.Keywords
Funding Information
- Division of Computer and Network Systems (CNS-1314721)
- Division of Computing and Communication Foundations (CCF-0964474)
This publication has 14 references indexed in Scilit:
- Scalable, high performance ethernet forwarding with CuckooSwitchPublished by Association for Computing Machinery (ACM) ,2013
- High-Performance Design of Hadoop RPC with RDMA over InfiniBandPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Workload analysis of a large-scale key-value storePublished by Association for Computing Machinery (ACM) ,2012
- SILTPublished by Association for Computing Machinery (ACM) ,2011
- A case for RDMA in cloudsPublished by Association for Computing Machinery (ACM) ,2011
- Benchmarking cloud serving systems with YCSBPublished by Association for Computing Machinery (ACM) ,2010
- Hopscotch HashingLecture Notes in Computer Science, 2008
- Can Memory-Less Network Adapters Benefit Next-Generation InfiniBand Systems?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- High Performance RDMA-Based MPI Implementation over InfiniBandInternational Journal of Parallel Programming, 2004
- Cuckoo hashingJournal of Algorithms, 2004