UNIT-V

UNIT – V SYLLABUS
Association Analysis: Basic Concepts and Algorithms: Problem Defecation,
Frequent Item Set generation, Rule generation, compact representation of
frequent item sets, FP-Growth Algorithm. (Tan &Vipin)

Association Analysis
Association mining aims to extract interesting correlations, frequent patterns,
associations or casual structures among sets of items or objects in transaction databases,
relational database or other data repositories. Association rules are widely used in various
areas such as telecommunication networks, market and risk management, inventory control,
cross-marketing, catalog design, loss-leader analysis, clustering, classification, etc.

EXAMPLES:
Rule Form: BodyHead [Support, confidence]

Buys (X, “Computer”)  Buys (X, “Software”) [40%, 50%]

ASSOCIATION RULE: BASIC CONCEPTS:

 Given: (1) database of transaction, (2) each transaction is a list of items (purchased by
a customer in visit)
 Find: all rules that correlate the presence of one set of items with that of another set
of items.
 E.g., 98% of people who purchase tires and auto accessories also get
automotive services done.
 E.g., Market Basket Analysis  This process analyzes customer buying habits
by finding associations between the different items that customers place in
their “Shopping Baskets”. The discovery of such associations can help retailers
develop marketing strategies by gaining insight into which items are frequently
purchased together by customer.

APPLICATIONS:
 Maintenance agreement (what the store should do to boost maintenance agreement
sales)
 Home Electronics (what other products should the store stocks up?) 
 Attached mailing in direct marketing 
ASSOCIATION RULE:
An association rule is an implication expression of the form XY, where X and Y are
disjoint itemsets, i.e., X ∩ Y = ∅. The strength of an association rule can be measured in
terms of its support and confidence. Support determines how often a rule is applicable to a
given data set, while confidence determines how frequently items in Y appear in
transactions that contain X. The formal definition of these metrics are

Support, s(XY) = 𝜎(𝑋∪Y)

𝑁

Confidence, c(XY) = 𝜎(𝑋∪Y)

𝜎(𝑋)
 Data Warehousing and Data Mining

Why Use Support and Confidence? Support is an important measure because a rule that has
very low support may occur simply by chance. A low support rule is also likely to be
uninteresting from a business perspective because it may not be profitable to promote items
that customers seldom buy together. For these reasons, support is often used to eliminate
uninteresting rules.

Confidence, on the other hand, measures the reliability of the inference made by a
rule. For a given rule XY, the higher the confidence, the more likely it is for Y to be present
in transactions that contain X. Confidence also provides an estimate of the conditional
probability of Y given X.

Therefore, a common strategy adopted by many association rule mining algorithms is to

decompose the problem into two major subtasks:

1. Frequent Itemset Generation, whose objective is to find all the item- sets that satisfy
the minsupthreshold. These itemsets are called frequent itemsets.
2. Rule Generation, whose objective is to extract all the high-confidence rules from the
frequent itemsets found in the previous step. These rules are called strong rules.

Frequent Itemset Generation:

A lattice structure can be used to enumerate the list of all possible itemsets. Above
Figure shows an itemset lattice for I = {a, b, c, d, e}. In general, a data set that contains k
items can potentially generate up to 2k − 1 frequent itemsets, excluding the null set. Because
k can be very large in many practical applications, the search space of itemsets that need to
be explored is exponentially large.
Data Warehousing and Data Mining
To find frequent itemsets we have two algorithms,

a) Apriori Algorithm
b) FP-Growth
 Data Warehousing and Data Mining

a) APRIORI ALGORITHM:
Apriori is a seminal algorithm proposed by R. Agrawal and R. Srikant in 1994 for mining
frequent itemsets for Boolean association rules. The name of the algorithm is based on the
fact that the algorithm uses prior knowledge of frequent itemset properties, as we shall see
later. Apriori employs an iterative approach known as a level-wise search, where k- itemsets
are used to explore (k+1)-itemsets.

First, the set of frequent 1-itemsets is found by scanning the database to accumulate
the count for each item, and collecting those items that satisfy minimum support. The
resulting set is denoted by L1. Next, L1 is used to find L2, the set of frequent 2-itemsets, which
is used to find L3, and so on, until no more frequent k-itemsets can be found. The finding of
each Lk requires one full scan of the database.

To improve the efficiency of the level-wise generation of frequent itemsets, an

important property called the Apriori property is used to reduce the search space.

Apriori property: All nonempty subsets of a frequent itemset must also be frequent.

The Apriori property is based on the following observation. By definition, if an itemset

I does not satisfy the minimum support threshold, min sup, then I is not frequent, that is, P(I)<
min sup. If an item A is added to the itemset I, then the resulting itemset (i.e.,IUA) cannot
occur more frequently than I. Therefore, IUA is not frequent either, that is, P(IUA) < min sup.

This property belongs to a special category of properties called antimonotonicity in

the sense that if a set cannot pass a test, all of its supersets will fail the same test as well. It is
called antimonotonicity because the property is monotonic in the context of failing a test.

A two-step process is followed, consisting of join and prune actions.

1. The join step: To find Lk, a set of candidate k-itemsets is generated by joining Lk-1 with itself.
This set of candidates is denoted Ck.
2. The prune step: Ck is a superset of Lk, that is, its members may or may not be frequent, but
all of the frequent k-itemsets are included in Ck. A database scan to determine the count of
each candidate in Ck would result in the determination of Lk.
 Data Warehousing and Data Mining
EXAMPLE:
Data Warehousing and Data Mining

1. In the first iteration of the algorithm, each item is a member of the set of candidate 1-
itemsets, C1. The algorithm simply scans all of the transactions to count the number
of occurrences of each item.
2. Suppose that the minimum support count required is 2, that is, min sup = 2. (Here, we
are referring to absolute support because we are using a support count. The
corresponding relative support is 2/9 = 22%.) The set of frequent 1-itemsets, L1, can
then be determined. It consists of the candidate 1-itemsets satisfying minimum
support. In our example, all of the candidates in C1 satisfy minimum support.
3. To discover the set of frequent 2-itemsets, L2, the algorithm uses the join L1 ⋈ L1 to
generate a candidate set of 2-itemsets, C2. C2 consists of 2-itemsets. Note that no
candidates are removed from C2 during the prune step because each subset of the
candidates is also frequent.
4. Next, the transactions in D are scanned and the support count of each candidate
itemset in C2 is accumulated, as shown in the middle table of the second row in Figure
5. The set of frequent 2-itemsets, L2, is then determined, consisting of those candidate
2- itemsets in C2 having minimum support.
 Data Warehousing and Data Mining

6. The generation of the set of the candidate 3-itemsets, C3, is detailed in Figure From
the join step, we first get C3 = L2 ⋈ L2 = {{I1, I2, I3}, {I1, I2, I5}, {I1, I3, I5}, {I2, I3, I4}, {I2,
I3, I5}, {I2, I4, I5}} Based on the Apriori property that all subsets of a frequent itemset
must also be frequent, we can determine that the four latter candidates cannot
possibly be frequent. We therefore remove them from C3, thereby saving the effort of
unnecessarily obtaining their counts during the subsequent scan of D to determine L3.
7. The transactions in D are scanned to determine L3, consisting of those candidate 3-
itemsets in C3 having minimum support.
8. The algorithm uses L3 ⋈ L3 to generate a candidate set of 4-itemsets, C4. Although the
join results in {I1, I2, I3, I5}, itemset {I1, I2, I3, I5} is pruned because its subset

{I2, I3, I5} is not frequent. Thus, C 4 ≠Ø, and the algorithm terminates, having found all of
the frequent itemsets.

b) FP-GROWTH:

FP-growth (finding frequent itemsets without candidate generation). We reexamine

the mining of transaction database, D, of Table in previous Example using the frequent pattern
growth approach.

The first scan of the database is the same as Apriori, which derives the set of frequent
items (1-itemsets) and their support counts (frequencies). Let the minimum support count be

2. The set of frequent items is sorted in the order of descending support count. This resulting set
or list is denoted by L. Thus, we have L = {{I2:7}, {I1:6}, {I3:6}, {I4:2}, {I5:2}}

An FP-tree is then constructed as follows. First, create the root of the tree, labeled
with “null.” Scan database D a second time. The items in each transaction are processed in L
Data Warehousing and Data Mining
order (i.e., sorted according to descending support count), and a branch is created for each
transaction.
 Data Warehousing and Data Mining

The FP-tree is mined as follows. Start from each frequent length-1 pattern (as an initial
suffix pattern), construct its conditional pattern base (a “sub-database,” which consists of
the set of prefix paths in the FP-tree co-occurring with the suffix pattern), then construct its
(conditional) FP-tree, and perform mining recursively on the tree. The pattern growth is
achieved by the concatenation of the suffix pattern with the frequent patterns generated
from a conditional FP-tree.

Finally, we can conclude that frequent itemsets are {I2, I1, I5} and {I2, I1, I3}.

GENERATING ASSOCIATION RULES FROM FREQUENT I TEMSETS

Once the frequent itemsets from transactions in a database D have been found, it is
straightforward to generate strong association rules from them (where strong association
rules satisfy both minimum support and minimum confidence). This can be done using Eq. for
confidence, which we show again here for completeness:
Data Warehousing and Data Mining
Data Warehousing and Data Mining