Simultaneous equation systems for query processing on continuous-time data streams

Yanif Ahmad*, Olga Papaemmanouil, Uǧur Çetintemel, Jennie Rogers

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

We introduce Pulse, a framework for processing continuous queries over models of continuous-time data, which can compactly and accurately represent many real-world activities and processes. Pulse implements several query operators, including filters, aggregates and joins, that work by solving simultaneous equation systems, which in many cases is significantly cheaper than processing a stream of tuples. As such, Pulse translates regular queries to work on continuous-time inputs, to reduce computational overhead and latency while meeting user-specified error bounds on query results. For error bound checking, Pulse uses an approximate query inversion technique that ensures the solver executes infrequently and only in the presence of errors, or no previously known results. We first discuss the high-level design of Pulse, which we fully implemented in a stream processing system. We then characterise Pulse's behavior through experiments with real data, including financial data from the New York Stock Exchange, and spatial data from the Automatic Identification System for tracking naval vessels. Our results verify that Pulse is practical and demonstrates significant performance gains for a variety of workload and query types.

Original languageEnglish (US)
Title of host publicationProceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08
Pages666-675
Number of pages10
DOIs
StatePublished - Oct 1 2008
Event2008 IEEE 24th International Conference on Data Engineering, ICDE'08 - Cancun, Mexico
Duration: Apr 7 2008Apr 12 2008

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627

Other

Other2008 IEEE 24th International Conference on Data Engineering, ICDE'08
CountryMexico
CityCancun
Period4/7/084/12/08

Fingerprint

Query processing
Processing
Naval vessels
Identification (control systems)
Experiments

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Ahmad, Y., Papaemmanouil, O., Çetintemel, U., & Rogers, J. (2008). Simultaneous equation systems for query processing on continuous-time data streams. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08 (pp. 666-675). [4497475] (Proceedings - International Conference on Data Engineering). https://doi.org/10.1109/ICDE.2008.4497475
Ahmad, Yanif ; Papaemmanouil, Olga ; Çetintemel, Uǧur ; Rogers, Jennie. / Simultaneous equation systems for query processing on continuous-time data streams. Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. pp. 666-675 (Proceedings - International Conference on Data Engineering).
@inproceedings{c861603c31964ac0b2d601abe52ddfcb,
title = "Simultaneous equation systems for query processing on continuous-time data streams",
abstract = "We introduce Pulse, a framework for processing continuous queries over models of continuous-time data, which can compactly and accurately represent many real-world activities and processes. Pulse implements several query operators, including filters, aggregates and joins, that work by solving simultaneous equation systems, which in many cases is significantly cheaper than processing a stream of tuples. As such, Pulse translates regular queries to work on continuous-time inputs, to reduce computational overhead and latency while meeting user-specified error bounds on query results. For error bound checking, Pulse uses an approximate query inversion technique that ensures the solver executes infrequently and only in the presence of errors, or no previously known results. We first discuss the high-level design of Pulse, which we fully implemented in a stream processing system. We then characterise Pulse's behavior through experiments with real data, including financial data from the New York Stock Exchange, and spatial data from the Automatic Identification System for tracking naval vessels. Our results verify that Pulse is practical and demonstrates significant performance gains for a variety of workload and query types.",
author = "Yanif Ahmad and Olga Papaemmanouil and Uǧur {\cC}etintemel and Jennie Rogers",
year = "2008",
month = "10",
day = "1",
doi = "10.1109/ICDE.2008.4497475",
language = "English (US)",
isbn = "9781424418374",
series = "Proceedings - International Conference on Data Engineering",
pages = "666--675",
booktitle = "Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08",

}

Ahmad, Y, Papaemmanouil, O, Çetintemel, U & Rogers, J 2008, Simultaneous equation systems for query processing on continuous-time data streams. in Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08., 4497475, Proceedings - International Conference on Data Engineering, pp. 666-675, 2008 IEEE 24th International Conference on Data Engineering, ICDE'08, Cancun, Mexico, 4/7/08. https://doi.org/10.1109/ICDE.2008.4497475

Simultaneous equation systems for query processing on continuous-time data streams. / Ahmad, Yanif; Papaemmanouil, Olga; Çetintemel, Uǧur; Rogers, Jennie.

Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 666-675 4497475 (Proceedings - International Conference on Data Engineering).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Simultaneous equation systems for query processing on continuous-time data streams

AU - Ahmad, Yanif

AU - Papaemmanouil, Olga

AU - Çetintemel, Uǧur

AU - Rogers, Jennie

PY - 2008/10/1

Y1 - 2008/10/1

N2 - We introduce Pulse, a framework for processing continuous queries over models of continuous-time data, which can compactly and accurately represent many real-world activities and processes. Pulse implements several query operators, including filters, aggregates and joins, that work by solving simultaneous equation systems, which in many cases is significantly cheaper than processing a stream of tuples. As such, Pulse translates regular queries to work on continuous-time inputs, to reduce computational overhead and latency while meeting user-specified error bounds on query results. For error bound checking, Pulse uses an approximate query inversion technique that ensures the solver executes infrequently and only in the presence of errors, or no previously known results. We first discuss the high-level design of Pulse, which we fully implemented in a stream processing system. We then characterise Pulse's behavior through experiments with real data, including financial data from the New York Stock Exchange, and spatial data from the Automatic Identification System for tracking naval vessels. Our results verify that Pulse is practical and demonstrates significant performance gains for a variety of workload and query types.

AB - We introduce Pulse, a framework for processing continuous queries over models of continuous-time data, which can compactly and accurately represent many real-world activities and processes. Pulse implements several query operators, including filters, aggregates and joins, that work by solving simultaneous equation systems, which in many cases is significantly cheaper than processing a stream of tuples. As such, Pulse translates regular queries to work on continuous-time inputs, to reduce computational overhead and latency while meeting user-specified error bounds on query results. For error bound checking, Pulse uses an approximate query inversion technique that ensures the solver executes infrequently and only in the presence of errors, or no previously known results. We first discuss the high-level design of Pulse, which we fully implemented in a stream processing system. We then characterise Pulse's behavior through experiments with real data, including financial data from the New York Stock Exchange, and spatial data from the Automatic Identification System for tracking naval vessels. Our results verify that Pulse is practical and demonstrates significant performance gains for a variety of workload and query types.

UR - http://www.scopus.com/inward/record.url?scp=52649170247&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=52649170247&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2008.4497475

DO - 10.1109/ICDE.2008.4497475

M3 - Conference contribution

SN - 9781424418374

T3 - Proceedings - International Conference on Data Engineering

SP - 666

EP - 675

BT - Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08

ER -

Ahmad Y, Papaemmanouil O, Çetintemel U, Rogers J. Simultaneous equation systems for query processing on continuous-time data streams. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 666-675. 4497475. (Proceedings - International Conference on Data Engineering). https://doi.org/10.1109/ICDE.2008.4497475