- So far, we have covered a method to find
all item sets
that have
sufficient support
- I.e.:
- We can find all item sets S
such that:
-
Freq(S) ≥
θ × N
- We can find all item sets S
such that:
- Note that all frequent item sets
in the result will
satify the
minimum support requirement
of the association rule
- We must still satisfy the
minimum confidence requirement
to obtain "useful" association rules
(A rule that has low confidence value is not useful because the rule is rarely valid)
- Suppose Y is
a frequent item set
And |Y| = k
- The number of potential
association rules that can be
obtained from Y is:
- number of possible association rule obtained from Y = 2k - 2
This number excludes these meaningless rules:
- ∅ &rArr Y
-
This rule is absurd, from
no information (∅),
we can conclude something
(Y)...
Can only happen when
Y is
known to be true
- Y &rArr ∅
-
This rule is trivially true.
- Example: frequent item set
{1, 2, 3}
Possible association rules that can be generated:
- {1} ⇒ {2, 3}
- {2} ⇒ {1, 3}
- {3} ⇒ {1, 2}
- {1. 2} ⇒ {3}
- {1, 3} ⇒ {2}
- {2, 3} ⇒ {1}
- The naive way to
obtain all association rules
is to
enumerate all
association rules and
check
if the minimum confidence level
is satisfied:
Input: F = all frequent item sets for ( each (frequent) item set S ∈ F ) do { for ( each proper subset X &sub S ) do { Y = S - X; // X is the antecedent // Y is the consequence if ( ( Freq( S (= X∪Y) ) / Freq(X) ) ≥ min_confidence ) { print X ⇒ Y; } } }
- This algorithm is
not practical because
number of potential
association rules is
exponentially large
(2k - 2)
This requires a smarter way to reduce the number of checks
- The following property
can be used to
prune off the
large number of
potential
association rules:
- If an association rule
X ⇒ S − X
has less than the
minimum confidence threshold,
i.e.:
- Freq(S) / Freq(X) < min_confidence
then:
- &forall X' ⊆ X: Freq(S) / Freq(X') < min_confidence
Proof:
- If X' ⊆ X,
then a transaction than
contains all the items in
X will
also contain
all the items in
X'
Therefore:
- Freq(X')
≥ Freq(X)
- Or: 1/Freq(X') ≤ 1/Freq(X)
- Freq(X')
≥ Freq(X)
- Therefore:
- Freq(S) / Freq(X') ≤ Freq(S) / Freq(X) < min_confidence
- If an association rule
X ⇒ S − X
has less than the
minimum confidence threshold,
i.e.:
- Example:
- Suppose S = {1,2,3,4}
- Suppose the rule:
-
{1,2,3} ⇒ {4}
does not have minimal confidence support
- Then these rules will not have
minimal confidence support:
- {1,2} ⇒ {3,4}
- {1,3} ⇒ {2,4}
- {1,4} ⇒ {2,3}
- {1} ⇒ {2,3,4}
- {2} ⇒ {1,3,4}
- {3} ⇒ {1,2,4}
- Suppose S = {1,2,3,4}
- This relationship can be
graphically shown in a
lattice-structure:
- Assume the frequent item set
is:
- {a,b,c,d}
- First, find
association rules with
a single value for consequence:
- Confidence({abc} ⇒ {d}) < min_confidence
- Confidence({abd} ⇒ {c}) ≥ min_confidence
- Confidence({acd} ⇒ {b}) ≥ min_confidence
- Confidence({bcd} ⇒ {a}) < min_confidence
- The Confidence-Based
pruning tells us that:
- You should not include
d and
a
in the search for rules of the form:
- ..... ⇒ {xy}
x ≠ a, x ≠ d, y ≠ a and y ≠ d
- You should not include
d and
a
in the search for rules of the form:
- The search for larger consequence sets
can be restricted
to unions of the sets:
- {c}
- {b}
- The association rule generation
algorithm uses a
level-wise approach:
- First, find rules with at least
minimum confidence
of the form:
- .... ⇒ {x} (single attribute in the consequence)
- Using rules found
in step 1,
find rules with at least
minimum confidence
of the form:
- .... ⇒ {x,y} (two attributes in the consequence)
(By using only the rules found in step 1, we are applying Confidence-Based pruning)
- And so on.
- First, find rules with at least
minimum confidence
of the form:
- Example:
- Suppose we found the following rules have
high confidence:
- {acd} ⇒ {b}
- {abd} ⇒ {c}
- In the next iteration,
we will only test the
following rule:
- {ad} ⇒ {bc}
- All other rules have been elminated by Confidence-Based pruning
- Suppose we found the following rules have
high confidence:
- Driver routine for the Rule-Generation algorithm:
for ( each frequent k-itemset fk (k ≥ 2) ) do { /* -------------------------------------------------- Compute association rules with 1-item consequence -------------------------------------------------- */ H1 = ∅ // All assoc. rules with 1-item consequence for ( each x ∈ fk ) do if ( ( Freq( fk ) / Freq({x}) ) ≥ min_confidence ) Add {x} to H1; /* -------------------------------------------------------- Find association rules with 2 or more items consequence -------------------------------------------------------- */ Apriori-GenRules( fk, H1 ); }
- Apriori-GenRules(fk, Hm):
-
generate association rules
using the
frequent item set fk
and
input set
association rules Hm
with m-item itemset consequences
Apriori-GenRules( fk, Hm ) { k = | fk |; // # items in fk m = | { x | x ∈ Hm } |; // m = size of the consequence /* ----------------------------------------- Stop searching when m = k - 1 (avoid generating the rule: ∅ &rArr fk ----------------------------------------- */ if ( m == k - 1 ) return; /* ---------------------------------- Generate "candidate consequences" ---------------------------------- */ Hm+1 = Apriori-Gen( Hm ); // See: click here /* --------------------------------------------------- Delete consequences with less than min threshold --------------------------------------------------- */ for ( each hm+1 ∈ Hm+1 ) do { conf = Freq(fk) / Freq(fk - hm+1); if ( conf ≥ min_confidence ) { output rule (fk - hm+1) ⇒ hm+1; } else { delete hm+1 from Hm+1; } } /* ---------------------------------------- Find higher degree consequence rules ---------------------------------------- */ if ( Hm+1 != ∅ ) { Apriori-GenRules( fk, Hm+1 ); } }