Stream processing in data-driven computational science
- 1 January 2006
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2006 7th IEEE/ACM International Conference on Grid Computing
Abstract
The use of real-time data streams in data-driven computational science is driving the need for stream processing tools that work within the architectural framework of the larger application. Data stream processing systems are beginning to emerge in the commercial space, but these systems fail to address the needs of large-scale scientific applications. In this paper we illustrate the unique needs of large-scale data driven computational science through an example taken from weather prediction and forecasting. We apply a realistic workload from this application against our Calder stream processing system to determine effective throughput, event processing latency, data access scalability, and deployment latencyKeywords
This publication has 13 references indexed in Scilit:
- Calder query grid service: insights and experimental evaluationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- The 8 requirements of real-time stream processingACM SIGMOD Record, 2005
- Evaluation of Rate-Based Adaptivity in Asynchronous Data Stream JoinsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Aurora: a new model and architecture for data stream managementThe VLDB Journal, 2003
- Leveraging run time knowledge about event rates to improve memory utilization in wide area data stream filteringPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- TelegraphCQPublished by Association for Computing Machinery (ACM) ,2003
- Dynamic querying of streaming data with the dQUOB systemIEEE Transactions on Parallel and Distributed Systems, 2003
- Monitoring XML data on the WebACM SIGMOD Record, 2001
- EddiesPublished by Association for Computing Machinery (ACM) ,2000
- An event-based architecture definition languageIEEE Transactions on Software Engineering, 1995