Jumpgate
- 14 June 2021
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM) in Proceedings of the 14th ACM International Conference on Systems and Storage
Abstract
Network-connected accelerators (NCA), such as programmable switches, ASICs, and FPGAs can speed up operations in data analytics. But so far, integration of NCAs into data analytics systems required manual effort. We present Jumpgate, a system that simplifies integration of existing NCA code into data analytics systems, such as Apache Spark or Presto. Jumpgate places most of the integration code into the analytics system, which needs to be written once, leaving NCA programmers to write only a couple hundred lines of code to integrate new NCAs. Jumpgate relies on dataflow graphs that most analytics systems use internally, and takes care of the invocation of NCAs, the necessary format conversion, and orchestration of their execution via novel staged network pipelines. Our implementation of Jumpgate in Apache Spark made it possible, for the first time, to study the benefits and drawbacks of using NCAs across the entire range of queries in the TPC-DS benchmark. Since we lack hardware that can accelerate all analytics operations, we implemented NCAs in software. We report on how and when analytics workloads will benefit from NCAs to motivate future designs.Keywords
This publication has 29 references indexed in Scilit:
- High Performance Packet Processing with FlexNICPublished by Association for Computing Machinery (ACM) ,2016
- How good are query optimizers, really?Proceedings of the VLDB Endowment, 2015
- On-the-Fly Pipeline ParallelismACM Transactions on Parallel Computing, 2015
- MusketeerPublished by Association for Computing Machinery (ACM) ,2015
- NaiadPublished by Association for Computing Machinery (ACM) ,2013
- Discretized streamsPublished by Association for Computing Machinery (ACM) ,2013
- Query processing on smart SSDsPublished by Association for Computing Machinery (ACM) ,2013
- OpenStack: Toward an Open-source Solution for Cloud ComputingInternational Journal of Computer Applications, 2012
- DremelProceedings of the VLDB Endowment, 2010
- TPC-DS, taking decision support benchmarking to the next levelPublished by Association for Computing Machinery (ACM) ,2002