---

STREAM: The Continuous Query Language (CQL)

---

• Introduction to STREAM
  • STREAM is the Stanford Data Stream Management System

    (Looks like an acronym... but not easy to "fit" the description)

    ---

  • Unlike Aurora that uses an graphical tool to allow user to construct stream queries, STREAM uses a stream language construct

  • STREAM builds on top of SQL and introduce new language constructs to accomodate data streams

    ---

  • STREAM recognizes two data types:

    Relation - this is the classical relational database "relation" stream - - this is the new data type introduced in STREAM

    ---

  • STREAM operators can operate of either data type.

    There are 3 kinds of STREAM operators:

    The 3 types of STREAM operators in more details:

    relation-to-relation operator: takes one or more relations as input and produces a relation as output stream-to-relation operator: takes one or more streams as input and produces a relation as output relation-to-stream operator: takes one or more relations as input and produces a stream as output

  ---

  ---

• What exactly is a stream and converting between a relation and a stream
  • Relation:

    A relation R is a time-varying collection (bag) of tuples The content of a relation can change when tuples are inserted, deleted or updated

    ---

  • Stream:

    A stream S is a unbounded collection (bag) of time-stamped tuples The time stamp of a tuple is the logical arrival time of the tuple

  ---

  ---

• Operators in CQL
  • As mentioned above, the CQL (Continuous Query Language) has 3 kinds of operators:

    Relation-to-relation Relation-to-stream Stream-to-relation

  ---

  ---

• Relation-to-relation operators
  • CQL uses then relational operators in SQL

  ---

  ---

• Relation-to-stream operators
  • There are 3 relation-to-stream operators in CQL (these operators take one or more relations as input and outputs a stream):

    Istream(): outputs (τ, t) whenever a tuple t is inserted into the input relation at time τ Dstream(): outputs (τ, t) whenever a tuple t is deleted from the input relation at time τ Rstream(): outputs (τ, t) whenever a tuple t is updated in the input relation at time τ

  ---

  ---

• Stream-to-relation operators
  • The stream-to-relation operators are all based on the concept of a sliding window over a stream...

    The relation is obtained from a stream by extracting the set (collection) of tuple in the window

    ---

  • There are 3 types of sliding windows defined in CQL:

    Tuple-based sliding window of size N on a stream S (taken at time τ The resulting relation R(τ) is the set (collection) of N tuples in S with the largest time stamps <= τ) Graphically: tuples ordered by theire time stamps: R(τ) N tuples +---------------------+ | | x x x x x x x x x -----> stream S ^ | τ How to express this window in CQL: [Rows N] Special form: [Rows Unbounded] denotes an append-only window Time-based sliding window of size N on a stream S (taken at time τ) The resulting relation R(τ) is the set (collection) of tuples with time stamps between (τ - N, τ) Graphically: Time --------------------------------------------------> ^ ^ | | τ-N τ | | v v tuple arrivals: x x x x x x x x x x | | +-------------------------+ R(τ) How to express this window in CQL: [Range N] Special form: [Now] denotes a window with width 0 - or the current tuple Partitioned sliding window of size N on a stream S based on the attributes A1, A2, ... Ak (taken at time τ) This is a more general form of tuple-based sliding window The input tuples are first partitioned according to their attribute values in A1, A2, ... Ak (I.e., the input stream is logically partititioned into a number of sub-streams according to their attribute values) Then the tuple-based sliding window of size N is computed on each sub-stream The resulting relation R(τ) is the union of the tuples from all sub-streams Graphically: input stream: (N = 2) +------+ | | ------> a1 a2 a3 ---- / \ / \ a1 b1 a2 c1 c2 b2 a3 c3 .... -------> b1 b2 ------ --> a2 a3 b1 b2 c2 c3 \ / \ / ------> c1 c2 c3 ----- How to express this window in CQL: [Partition By A1, .., Ak Rows N]

  ---

  ---

• CQL query 1
  • The style of CQL resembles SQL

  • Consider the following CQL query:

    SELECT Istream(*) FROM S [Rows Unbounded] WHERE S.A > 10

    ---

  • Execution of the above query in STREAM:

    The source of the query is the stream S The stream-to-relation operator [Rows Unbounded] converts the input stream to a relation (append only set of tuples) Then the relation-to-relation filter S.A > 10 is applied; resulting in another relation Finally, the Istream(*) relation-to-stream operator converts the output (back) to a stream

  ---

  ---

• CQL query 2
  • Consider the following query:

    SELECT * FROM S1 [Range 1000], S2 [Range 2 Minutes] WHERE S1.A = S2.A AND S1.A > 10

    ---

  • Execution of the above query in STREAM:

    The sources of the query are stream S1 and stream s2 Stream S1 is converted into a relation consisting of the 1000 most recent tuples of S1 by [Range 1000] operator Stream S2 is also converted into a relation consisting of the recent tuples arriving in the last 2 min of S2 by [Range 2 Minutes] operator The 2 relations are joined using the condition S1.A = S2.A and the joined tuples are selected using S1.A > 10 The result remains a relation (because no relation-to-stream operator was used in the SELECT clause)

  ---

  ---

• CQL query 3
  • Consider the query:

    SELECT Rstream(S.A, S.B) FROM S [Now], R WHERE S.A = R.A

    ---

  • Execution of the above query in STREAM:

    The sources of the query are: stream S (because it is followed by a stream-to-relation) and relation R (because it is not followed by a stream-to-relation) The [Now] operator converts the arriving tuple of S into a relation This relation (consiting of one tuple) is joined with R based on S.A = R.A The result is converted into a stream (This query probes the relation R based in data in the input stream S)