Assessment of Classification Algorithms in Artificial Intelligence

In the area of artificial learners, not much research on the question of an appropriate description of artificial learner's (empirical) performance has been conducted. The optimal solution of describing a learning problem would be a functional dependency between the data, the learning algorithm's internal specifics and its performance. Unfortunately, a general, restrictions-free theory on performance of arbitrary artificial learners has not been developed yet. This work addresses the problem of measuring and observing the artificial learners, specifically the decision trees produced by the C4.5 algorithm. A procedure for measuring the learning progress, called adaptive incremental k-fold cross-validation is presented, together with other tools and techniques needed to observe artificial learners on their course of learning. Early observations can be used to forecast the future performance of a learner based on a small training sample.