Dynamic querying of streaming data with the dQUOB system
- 22 April 2003
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 14 (4), 422-432
- https://doi.org/10.1109/tpds.2003.1195413
Abstract
Data streaming has established itself as a viable communication abstraction in data-intensive parallel and distributed computations, occurring in applications such as scientific visualization, performance monitoring, and large-scale data transfer. A known problem in large-scale event communication is tailoring the data received at the consumer. It is the general problem of extracting data of interest from a data source, a problem that the database community has successfully addressed with SOL queries, a time tested, user-friendly way for noncomputer scientists to access data. By leveraging the efficiency of query processing provided by relational queries, the dQUOB system provides a conceptual relational data model and SOL query access over streaming data. Queries can be used to extract data, combine streams, and create new streams. The language augments queries with an action to enable more complex data transformations such as Fourier transforms. The dQUOB system has been applied to two large-scale distributed applications: a safety critical autonomous robotics simulation and scientific software visualization for global atmospheric transport modeling. In this paper, we present the dQUOB system and the results of performance evaluation undertaken to assess its applicability in data-intensive wide-area computations, where the benefit of portable data transformation must be evaluated against the cost of continuous query evaluation.Keywords
This publication has 30 references indexed in Scilit:
- Design, implementation, and performance of an extensible toolkit for resource prediction in distributed systemsIEEE Transactions on Parallel and Distributed Systems, 2006
- GriPhyN and LIGO, building a virtual data Grid for gravitational wave scientistsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Software approach to hazard detection using on-line analysis of safety constraintsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- A component based services architecture for building distributed applicationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Event services for high performance computingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Design and evaluation of a wide-area event notification serviceACM Transactions on Computer Systems, 2001
- The data grid: Towards an architecture for the distributed management and analysis of large scientific datasetsJournal of Network and Computer Applications, 2000
- ACTIVE I/O STREAMS FOR HETEROGENEOUS HIGH PERFORMANCE COMPUTINGPublished by World Scientific Pub Co Pte Ltd ,2000
- OBJECT-RELATIONAL QUERIES INTO MULTIDIMENSIONAL DATABASES WITH THE ACTIVE DATA REPOSITORYParallel Processing Letters, 1999
- Distance visualization: data exploration on the gridComputer, 1999