Slideshow:
Strength and weakness of the
one-pass algorithms
Intro to the 2-pass
algorithms
Categories of 2-pass algorithms:
Overview of the
2-pass
algorithms
Overview of the
2-pass
algorithms
Pass 1: sort/hash the input relation into a number of smaller (chunks) relations
Pass 2: run a (possibly modified) one-pass algorithm on each (pair) of smaller (chunks) relation(s)
- 2-pass algorithms:
- Based of
hashing
- Based of sorting
- Based of
hashing
- Fact:
- One-pass algorithms are
the most efficient
algorithms to implement
a relational algebra operation
- However,
many One-pass algorithms
have
high memory requirments:
M ≥ B(S) (where S is the smaller relation)
If neither relation can fit in memory (buffers), then the one-pass algorithm will not perform efficiently
(Due to trashing: buffers must be moved back and forth between main memory and disk)
- One-pass algorithms are
the most efficient
algorithms to implement
a relational algebra operation
- We will now present
2 classes of
2-pass algorithms
with the
following
minimum buffer requirement:
------------- M ≥ \/ B(R) + B(S) buffers
- Characteristics of
all these
2-pass algorithms:
- Pass 1:
- Read one or both
input relation and
process the tuples:
-
Hash (= "categorize") the
tuples
into
(M−1) buckets
or
- Sort the input relations into chunks of M blocks
-
Hash (= "categorize") the
tuples
into
(M−1) buckets
or
-
Write the result
(or the sort or
hash
operation)
to disk
(to free up memory)
- Read one or both
input relation and
process the tuples:
- Pass 2:
- Read the
processed result
from
disk
And use the processed result to performed the required operation
- Read the
processed result
from
disk
Graphically:
- Pass 1: