High-Performance Design of Hadoop RPC with RDMA over InfiniBand
- 1 October 2013
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 641-650
- https://doi.org/10.1109/icpp.2013.78
Abstract
Hadoop RPC is the basic communication mechanism in the Hadoop ecosystem. It is used with other Hadoop components like MapReduce, HDFS, and HBase in real world data-centers, e.g. Facebook and Yahoo!. However, the current Hadoop RPC design is built on Java sockets interface, which limits its potential performance. The High Performance Computing community has exploited high throughput and low latency networks such as InfiniBand for many years. In this paper, we first analyze the performance of current Hadoop RPC design by unearthing buffer management and communication bottlenecks, that are not apparent on the slower speed networks. Then we propose a novel design (RPCoIB) of Hadoop RPC with RDMA over InfiniBand networks. RPCoIB provides a JVM-bypassed buffer management scheme and utilizes message size locality to avoid multiple memory allocations and copies in data serialization and deserialization. Our performance evaluations reveal that the basic ping-pong latencies for varied data sizes are reduced by 42%-49% and 46%-50% compared with 10GigE and IPoIB QDR (32Gbps), respectively, while the RPCoIB design also improves the peak throughput by 82% and 64% compared with 10GigE and IPoIB. As compared to default Hadoop over IPoIB QDR, our RPCoIB design improves the performance of the Sort benchmark on 64 compute nodes by 15%, while it improves the performance of CloudBurst application by 10%. We also present thorough, integrated evaluations of our RPCoIB design with other research directions, which optimize HDFS and HBase using RDMA over InfiniBand. Compared with their best performance, we observe 10% improvement for HDFS-IB, and 24% improvement for HBase-IB. To the best of our knowledge, this is the first such design of the Hadoop RPC system over high performance networks such as InfiniBand.Keywords
This publication has 8 references indexed in Scilit:
- Understanding the communication characteristics in HBase: What are the fundamental bottlenecks?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Memcached Design on High Performance RDMA Capable InterconnectsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Can MPI Benefit Hadoop and MapReduce Applications?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Unifying UPC and MPI runtimesPublished by Association for Computing Machinery (ACM) ,2010
- Benchmarking cloud serving systems with YCSBPublished by Association for Computing Machinery (ACM) ,2010
- The Hadoop Distributed File SystemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Differential serialization for optimized SOAP performancePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Efficient Java RMI for parallel programmingACM Transactions on Programming Languages and Systems, 2001