Classifier Learning for Imbalanced Data: a Comparison of Knn, Svm, and Decision Tree Learning

This work discusses the theoretical abilities ofthree commonly used classifier learning methods andoptimization techniques to cope with characteristicsof real-world classification problems, morespecifically varying misclassification costs,imbalanced data sets and varying degrees of hardnessof class boundaries. From these discussions a universally applicableoptimization framework is derived that successfullycorrects the error-based inductive bias of classifierlearning methods on image data within the domain ofmedical diagnosis. The framework was designed considering several pointsfor improvement of common optimization techniques,such as the modification of the optimizationprocedure for inducer-specific parameters, themodification of input data by an arcing algorithm,and the combination of classifiers according tolocally-adaptive, cost-sensitive voting schemes. The framework is designed to make the learningprocess cost-sensitive and to enforce more balancedmisclassification costs between classes. Results onthe evaluated domain are promising, while furtherimprovements can be expected after some modificationsto the framework.