Example Selection Measure:
Information Entropy
Let p(Cij) = Prob(Cj|A=ai) = Cij/Si
Entropy of class distribution over partition:
Let p(Cij) = Prob(Cj|A=ai) = Cij/Si
Entropy of class distribution over partition:
Attribute A with values { a1, a2, ..., ar }
Entropy(S
i
) max when all C
ij
are equal;
Entropy(S
i
) min when for some i,j C
ij
=S
i
