GridBatch: Cloud Computing for Large-Scale Data-Intensive Batch Applications
- 1 May 2008
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 295-305
- https://doi.org/10.1109/ccgrid.2008.30
Abstract
To be competitive, enterprises are collecting and analyzing increasingly large amount of data in order to derive business insights. However, there are at least two challenges to meet the increasing demand. First, the growth in the amount of data far outpaces the computation power growth of a uniprocessor. The growing gap between the supply and demand of computation power forces Enterprises to parallelize their application code. Unfortunately, parallel programming is both time-consuming and error-prone. Second, the emerging Cloud Computing paradigm imposes constraints on the underlying infrastructure, which forces enterprises to rethink their application architecture. We propose the GridBatch system, which aims at solving large-scale data-intensive batch problems under the Cloud infrastructure constraints. GridBatch is a programming model and associated library that hides the complexity of parallel programming, yet it gives the users complete control on how data are partitioned and how computation is distributed so that applications can have the highest performance possible. Through a real client example, we show that GridBatch achieves high performance in Amazon's EC2 computing Cloud.Keywords
This publication has 5 references indexed in Scilit:
- Map-reduce-mergePublished by Association for Computing Machinery (ACM) ,2007
- DryadPublished by Association for Computing Machinery (ACM) ,2007
- Interpreting the Data: Parallel Analysis with SawzallScientific Programming, 2005
- The Imagine Stream ProcessorPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- The Google file systemPublished by Association for Computing Machinery (ACM) ,2003