Need
for Data Mining Tools
-
Human analysis breaks down with volume
and dimensionality
-
How quickly can you digest 1 million records, with 100
fields each?
-
High rate of growth, changing underlying
source
-
What is typically done by non-statisticians?
-
select a few fields (usually 2-3 out of 50-100), attempt
to visualize models/separators or fit simple models
-
What about statistical tools?
-
do not scale to large databases
-
are not easy to use or require significant analysis
expertise
