|Application domain:||market research|
|Dataset size:||31Kb Prolog file, 100 examples|
|Data format:||Prolog facts, ACL input format|
|Systems Used:||ACL, C4.5, FOIL|
The data set has been represented in Prolog: each interviewed person is assigned a different numeric constant. For each attribute of the drink, the range from 1 to 5 was represented using three predicates: one for answers 1 and 2, one for answer 3 and one for answer 4 and 5. Questions about customer's personal tastes were represented using one predicate.
Overall, 100 persons were interviewed, 52 answered with 4 or 5 to the question ``would you buy the prodcut ?'' (e.g. postby(16)), 32 answered with 1 or 2 (e.g. postnotbuy(24)) and 16 do not know (e.g. postdkbuy(31)). For some attributes the information is incomplete: flavour (37 incomplete cases), refreshing (40), like delicious (75), like natural (68), like apparent fruit (89), dislike strong sour (88) and dislike nothing (68).
Some questions are unanswered or have don't care answers and these have
been treated as incomplete information in the background. Out of 24
background predicates, 8 are incomplete with degree of incompleteness
from 37% (i.e. 37 people out of loo have not answered or have answered
don't care) up to 89%. The incomplete background predicates have been
considered as abducibles and integrity constraints have been introduced
in order to avoid the abduction of two different answers for the same
question. For example, for the question of "overall flavour" we have the
following constraint on the abducible predicates that record answers to
<-- goodflavouroverall(X), poor flavouroverall(X).
The experiments were performed using only the first phase of ACL, called Intermediate-ACL. In this phase, it is learned an abductive theory containing only new rules, not new integrity constraints. The condition that the learned theory must satisfy can be rewritten as , where stands for the conjunction of all positive examples and of the negation of all negative examples1.
First we tried to learn the concept postbuy We used as negative examples postnotbuy. The first experiment was conducted using the information on all available attributes and gave the results:
Rules are followed by a maximum of 4 numbers in this form . is the number of positive examples covered by the rule with or without abduction. is the number of positive examples covered by the rule by using abduction (if absent is 0). is the number of negative examples covered by the rule, i.e. for which failed (if absent is 0, i.e. the rule is consistent). is the number of negative examples not covered by using abduction, i.e. succeeded with a non-empty explanation (if absent is 0).
It is interesting to investigate how the definitions learned for postbuy behave with respect to the 16 don't know examples, i.e. the examples for postdkbuy. The definition for postbuy covers 10 of the don't know examples, out of which 9 are covered with abduction. This means that, according to the definition learned for postbuy, in 10 cases out of 16 (around 60%) the indecisive customer will buy the product.
We learned as well the concepts postnotbuy and postdkbuy. Moreover, we performed a number of experiments on learning postbuy: without using abduction, using also negated literals in the body of rules, without using the flavour attribute, without using the sweetness attribute, without using both flavour and sweetness.
The figure following the rule represents the number of covered positive examples. Note that the previous definition does not cover 2 positive examples for postbuy.
ACL1, mFOIL, c4.5 (and FOIL) were run on this data. The performance of ACL1, mFOIL and c4.5 were compared by means of a 5-fold cross validation. The average results for accuracy and runtimes are shown in table 1. ACL1 has found theories that are, on average, more accurate than C4.5 and mFOIL with run times higher than C4.5 but lower than ~ mFOIL. In general, the dominant rules found by ACL1 (and the other systems) were judged to be meaningful by the experts.
The second phase of ACL was also run on this data to find constraints
which support the abductive rules and assumptions of ACLl. For example,
one of the constraints found was
<-- goodflavouroverall(X), higharoma(X)
which (partially) complements the available knowledge on goodflavouroverall(X). On average, the constraints found where again judged to be significant by experts.
Table 1: Performance on the drinks questionnaire data
|Accuracy||Run Times ( seconds )|