- One-pass algorithm:
while ( R has more data blocks ) { read data blocks to fill available buffer; Extract the projection attrs for each tuple in buffers; Move projected tuples to output }(Projection π does not remove duplicates...)
- Buffer utilization when
there are M buffers
available:
- Naive
parallel projection π:
- Each processor
executes the
uni-processor πattrs
locally on its
fragment
of relation R:
Problem:
- Projection can
(is often)
followed by
duplicate elimination δ:
- The operations will not cascade !!!
Example:
- Consider:
δ ( πattrs(R) )
Graphically:
- Each processor
executes the
uni-processor πattrs
locally on its
fragment
of relation R:
- Cause of the
problem:
- The hash distribution function
is based on the
entire tuple
- The projection πattrs( R ) has altered the structure of the tuples !!!
Consequently:
- We must perform post processing (= re-hashing) to cascade with the δ (and other) operator
- The hash distribution function
is based on the
entire tuple
- Fixing
parallel projection
algorithm
(so the operations will
cascade):
- Each processor
executes the
(naive)
uni-processor
πattrs operation
locally on its
fragment
of relation R:
- Re-distribute
(using hashing) the
tuples (according to the
projection attribute values):
- Each processor
executes the
(naive)
uni-processor
πattrs operation
locally on its
fragment
of relation R:
- Performance of the
parallel
πattrs:
- Performing
the uni-processor
projection πattrs on
the tuples:
- Read the
tuples from
disk:
B(R) blocks
- Write the
(projection) tuples to
disk:
B(R) blocks
(We assume the relation fragments are very large)
- Read the
tuples from
disk:
- Re-distributing the
tuples:
- Read all
disk blocks
(to perform the hashing re-distribution):
B(R) blocks
- Transfer
P-1 ----- of the disk blocks Pto other nodes (assuming uniform distribution of project attribute values)
- Write the
re-distribued tuples
back to disk:
0 blocks(Because the output of the last operation is not counted)
- Read all
disk blocks
(to perform the hashing re-distribution):
-
Graphically:
- Performing
the uni-processor
projection πattrs on
the tuples:
- Total amount of work:
Disk read/write: 3 B(R) blocks P-1 Transfered: ----- blocks P - Amount of work
per processor:
3 Disk read/write: --- B(R) blocks P P-1 Transfered: ----- blocks P2