- The Auroa System is a joint
work between
Brandeis University,
Brown University and
M.I.T.
Website: click here
- Is a system for
managing data streams
for
monitoring applications
- Monitoring applications
are applications that use:
- one or more information streams
- uses triggers to
detect certain events
(e.g., heart rate over certain threshold)
- uses imprecise data
- has real time requirements
- one or more information streams
- We saw that these applications are
poorly supported by
traditional database systems
- The following figure shows the
Aurora system architecture:
- A data stream is an
append-only
sequence of tuples
All tuples in a stream have the same structure ("schema")
- Each data source is
assumed to have a
unique source identifier
- The Aurora system
time stamps each
incoming data tuple
- Input data streams are
processed by operator boxes
Each operator box performs a simple operation on the data stream (e.g., select tuples with length > 100).
Operator boxes are connected into a connection network
- The connection network is constructed using a Graphical User Interface (the GUI will not be discussed)
- Aurora provides
seven (7) Stream Query Algebra
operators
(many of them have relational algebra
equivalents)
- Some of these operators are:
- filter
- applies a predicate to an incoming tuple
It works like the SELECT in Relational Algebra
- aggregate
- computes a function value across a
window of
incoming tuples
The accuracy of the aggregate function depends on the size of the window
- split
- sends one stream into the
input of
multiple boxes
- union
- merges
tuples from 2 input streams
into one stream
- re-sample
- is a delay mechanism to
make one tuple wait on the arrival of another tuple
It works like the SELECT in Relational Algebra
- filter
- applies a predicate to an incoming tuple
- Aurora supports
3 modes of query processing:
- Continuous query:
- A continuous query is
started at the
inception of
of input stream (i.e., when the input data is first received)
- Data flows continuously
through the processing pipeline in a
continuous query
- There is no need to store any tuple once they are processed.
- A continuous query is
started at the
inception of
of input stream (i.e., when the input data is first received)
- View:
- The output of a
view query is
materialized
(i.e., stored)
temporally.
- Applications can occassionally connect to a view query output to examine the output.
- The output of a
view query is
materialized
(i.e., stored)
temporally.
- Ad-hoc Query:
- An
Ad-Hoc query can be
started
after the inception
of the input streams
I.e., an ad-hoc query can be started while the input data has been coming in for some time.
- An Ad-Hoc query can
only be attached to a
connection point
(see below)
A connection point will buffer some amount of data (e.g., from the last T seconds )
- An Ad-Hoc query is
a continuous query,
but operating on input data starting from time
tnow - T
(The starting time is a little earlier because the connection point buffers T seconds of data)
- An
Ad-Hoc query can be
started
after the inception
of the input streams
- Continuous query:
- Connection points in
the Aurora system are
temporal buffers to
caches tuples that
"passes" through
the
(connection) point.
A "persistence specification" is (always) associated with a connection point that specify the duration for which the tuples are cached
- Purpose of the connection pints:
- One of the purpose of
connection points is
to allow
Ad-Hoc queries
to object input data
- Another purpose of
connection points is
to allow the Aurora System to
dynamically change
the interconnection processing network
Changing the path of data flow can sometime improve the performance of a stream query
- One of the purpose of
connection points is
to allow
Ad-Hoc queries
to object input data