- Greenwald and Khanna
and Zhang and Wang
algorithms can find the
ε-approximate quantile
in a data stream
from the beginning till the end
- Calders' algorithm find the
maximum window but this
window is
not pre-determined
- Some times, we are interested
in the
most recent items
in the stream...
Example:
- For example, what are the top 10% of movies rented in the last 3 months ?
- The following figure shows the effect of a
fixed sized window
W = 4 (fixed size)
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-+ Sum = 3
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----+ Sum = 8
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------+ Sum = 10
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 17
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 15
- There is also a variable sized window
The size can increase or decrease by one
The following figure shows the effect of a variable sized window
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-+ Sum = 3
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----+ Sum = 8
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------+ Sum = 10
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 17
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------------+ Sum = 18
Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Window can shrink Sum = 15
- Problem statement:
- Given a data stream
- And a window size W
- Maintain an
ε-approximate quantile summary
over the data items
in the current window of size W
- Given a data stream
- Greenwald and Khanna's
merge and prune
techniques can be used to
construct a
ε-approximate
quantile summary
for a sliding window over
a stream
- The input stream is divided
into smaller windows,
each of size
(ε/2)×W:
- We can use Greenwald & Khanna
algorithm
( click here)
to construct an
(ε/2)-approximate quantile
summary for
each sub-window:
- Sliding of the window:
- When one item
from a sub window
is outside of the window W,
the whole sub-window
is remove from the summary:
- When one item
from a sub window
is outside of the window W,
the whole sub-window
is remove from the summary:
- Facts:
- These block summaries
will cover:
- At least the last W - (ε/2)W data items
- At most the last W data items
- These block summaries
will cover:
- Claim:
- These block summaries is an ε-approximate summary for the last W items in the stream
Proof:
- Each block summary is a
ε/2-approximate summary
for its smaller window
- The combine result
of all block summaries
is a
ε/2-approximate summary
of
at least (1 - ε/2)W
of the last W
elements
in the stream
(There are at most (ε/2)W elements missing from the summaries !)
- In the worst case
all of these (ε/2)W
missing elements
falls at the start of the quantile
Even then, the error will be increased by (ε/2)W and the total error is at most εW