|Application domain:||Mutagenesis, regression-unfriendly|
|Dataset size:||2 556 facts|
|Systems Used:||STILL, DISTILL|
Most experiments regard the 188-compound dataset, known as regression-friendly since numerical regression obtains around 86% predictive accuracy.
The following experiments concern the 42 other compounds, i.e. the regression-unfriendly dataset which is considered to be harder than the 188-dataset (the best prediction rate reported on this dataset [#!SriKin96-ILP96!#] is 64%). This dataset involves 29 inactive compounds vs 13 active compounds.
The two parameters of stochastic subsumption and are respectively set to 300 and 3, and we focus on the influence of parameters and , where corresponds to perfect consistency and to maximal generality.
The table above summarizes the predictive accuracy of STILL on the test set, averaged on 25 independent selections of a 4-example test set distributed as the whole dataset. The third, fourth and fifth columns respectively give the percentage of correctly classified, unclassified and misclassified test examples. Column 6 gives the standard deviation of the predictive accuracy, Column 7 gives the total computational time (induction and classification of the test examples), in seconds on a HP-710 workstation.
The results obtained for reasonable values of and are satisfactory; the computational cost is negligible. One only regrets the high variance of the predictive accuracy.
|p||Average OK||Range||Average variance||time|
The two parameters of stochastic subsumption and are respectively set to 10 and 3, and we focus on the influence of the number of dimensions .
The above table summarizes the predictive accuracy of DISTILL, with the following experimental setting. A run corresponds to a 10-cross-fold validation; column 2 indicates the average predictive accuracy on 20 independent runs and column 3 indicates the range of variation of this predictive accuracy. The average variance of the cross-validation is given in column 4, and the computational time (induction of hypotheses, mapping all examples onto and k-NN classification of the test examples), in seconds on a HP-710 workstation.
The results obtained for sufficient values of () are satisfactory and degrade gracefully as decreases; the computational cost remains moderate. DISTILL obtains slightly better and overall more steady performances than STILL, whereas it involves one less parameter.