- Facts about parallel algorithms:
- Parallel algorithms
usually
perform more steps (work)
to complete the
(same) task as
serial algorithms
- Parallel algorithms does not do less work !!!
- Speed up achieved
in parallel algorithms are
due to:
- More processors doing the work !!!
- Parallel algorithms
usually
perform more steps (work)
to complete the
(same) task as
serial algorithms
- A key to
achieving
good performance in
parallel/distributed algorithms:
- Distribute the work load evenly among all processors !!!
- Problem:
- Find the maximum value in an array A[N] (N is very large)
- Serial algorithm:
- Processor scan
the entire array:
- Running time:
- N steps (comparisons)
- Processor scan
the entire array:
- Parallel algorithm with
P processors:
- Make each processor find the
maximum of a
subset of
array elements:
- Find the maximum among the
P maxima:
- Running time:
Find the maximum for each processor: Ni (parallel) Find the maximum between maxima: P Running time = max( Ni ) + P
- Make each processor find the
maximum of a
subset of
array elements:
-
Effect of the
work load distribution:
- Bad
work load distribution:
P-1 processor gets 1 array element 1 processor gets the rest
Find the max for each processor: 0, 0, ...., N-P+1 Find the max between maxima: P Running time: max( 0,0,..,N-P+1 ) + P = N-P+1 + P = N+1
- Optimum
work load distribution:
Each processor gets 1/P portion of all array elements
Find the max for each processor: N/P-1, N/P-1, ...., N/P-1 Find the max between maxima: P Running time: max( N/P-1, N/P-1, ...., N/P-1 ) + P = N/P+1 + PExample:
N = 1,000,000,000 array elements
Serial: running time = 999,999,999 steps (comparisons)
Parallel with P = 1,000 processors running time = 1,000,000,000/1,000 + 1 + 1,000 = 1,000,000 + 1 + 1,000 = 1,001,001 steps (comparisons)
- Bad
work load distribution:
- Distributing the
tuples of a
relation to
support
parallel processing:
- Data (= tuples) stored in a
relation is
distributed over
the disks:
- Data (= tuples) stored in a
relation is
distributed over
the disks:
- Parallel execution of
relational algebra operations:
- Each processor will
perform the
relational algebra operation
on its local segment
of the relation
Example: selection σ
- Each processor will
perform the
relational algebra operation
on its local segment
of the relation
- Fact:
- Running time of
relational algebra
operations is (usually)
- Proportional to (one of) the (input) relation
- Running time of
relational algebra
operations is (usually)
- Good
work load distribution in
data processing:
- Distribute each
relation equally
between all processors
Graphically:
- Distribute each
relation equally
between all processors
- Method to
distribute
the tuples of
relations over
processing nodes:
- Use a distribution hash function
Hdistr(tuple):
Note:
- The distribution hash function
uses the
- entire tuple
to avoid "hot spots" created by non-key (= duplicate) attributes
- Use a distribution hash function
Hdistr(tuple):
- Recall:
- A good hash function will hash a tuple in a bucket with equal probability
Therefore:
- The size of each relation segment will be approximately equal !!!
- Postscript:
- There are also additional benefits when we use a hash function to distribute tuples