- Joining on equality
of attribute values:
R = ( A, B ) S = ( A, C ) R ⋈A=A S
- Example:
- Similar to the
Matrix Multiplication
problem:
- The map( ) function
(really) only has
one input stream:
of the format ( keyi , valuei )
- The map( ) function
(really) only has
one input stream:
- The inputs of the
Join operation are:
- Tow (2)
input relations:
- Tow (2)
input relations:
- Therefore:
- We must insert a
pre-processing step
to:
- Convert the
input relations to
the form:
( key1 , value1 ) ( key2 , value2 ) ( key3 , value3 ) ...
Graphically:
- Convert the
input relations to
the form:
- We must insert a
pre-processing step
to:
- Pre-processing used for
join (⋈) operation:
Summary:
- The map( ) function:
- The map( ) performs
the following
transformation on
an input value:
(R, (a, b)) ---> ( a, (R, b)) (S, (a, c)) ---> ( a, (S, c))
Example:
- The map( ) performs
the following
transformation on
an input value:
- The shuffle mechanism of
MapReduce will
re-organize (group) the
map( ) output as
follows:
(In this example, the shuffle mechanism will form 3 groups)
- The reduce( ) function:
- The reduce( ) function
performs the
cartesian product on
each group:
a = key of the group for ( each tuple (R, b) ) do { for ( each tuple (S, c) ) do { output tuple: ( a, b, a, c ) ; } }
Example:
- The reduce( ) function
performs the
cartesian product on
each group: