KEEL: A software tool to assess evolutionary algorithms for Data Mining problems (regression, classification, clustering, pattern mining and so on)

dataset/studies/classification/fernandez-IEEETEC10.gif

A. Fernandez, S. García, J. Luengo, E. Bernadó-Mansilla, F. Herrera, Genetics-Based Machine Learning for Rule Induction: State of the Art, Taxonomy and Comparative Study. IEEE Transactions on Evolutionary Computation, Doi: 10.1109/TEVC.2009.2039140, In press (2010).

This webpage contains:

Abstract
Summary
Experimental study

Additional links:

External website associated to this paper.

Abstract:

The classification problem can be addressed by numerous techniques and algorithms, which belong to different paradigms of Machine Learning. In this work, we are interested in evolutionary algorithms, the so-called Genetics-Based Machine Learning algorithms. In particular, we will focus on evolutionary approaches that evolve a set of rules, i.e., evolutionary rule-based systems, applied to classification tasks, in order to provide a state of-the-art in this field.

This study has been done with a double aim: to present a taxonomy of the Genetics-Based Machine Learning approaches for rule induction, and to develop an empirical analysis both for standard classification and for classification with imbalanced data sets.

We also include a comparative study of the GBML methods with some classical non-evolutionary algorithms, in order to observe the suitability and high power of the search performed by evolutionary algorithms and the behaviour for the GBML algorithms in contrast to the classical approaches, in terms of classification accuracy.

Summary:

1. Introduction
2. Taxonomy of genetics-based machine learning algorithms for classification
3. Experimental framework
4. Analysis of the GBML algorithms for rule induction in standard classification
5. Analysis of the GBML algorithms for rule induction in imbalanced data sets
6. Discussion: Lessons learned and new challenges
7. Concluding remarks

Experimental study:

Algorithms analyzed: XCS, UCS, SIA, HIDER, CORE, OCEC, COGIN, GIL, Pitts-GIRLA, DMEL, GASSIST, OIGA, ILGA, DT-GA, Oblique-DT, TARGET, CART, AQ, CN2, C4.5, C4.5-Rules, Ripper.
Data sets used: ZIP file

Standard: [5-fcv] abalone, australian, balance, breast, bupa, car, cleveland, contraceptive, crx, dermatology, ecoli, flare, german, glass, haberman, heart, hepatitis, iris, lymphography, magic, new-thyroid, nursery, penbased, pima, ring, tic-tac-toe, vehicle, wine, wisconsin, zoo.

Imbalanced: [5-fcv] glass1, ecoli-0_vs_1, wisconsin, pima, iris0, glass0, yeast1, vehicle1, vehicle2, vehicle3, haberman, glass-0-1-2-3_vs_4-5-6, vehicle0, ecoli1, new-thyroid2, new-thyroid1, ecoli2, segment0, glass6, yeast3, ecoli3, page-blocks0, vowel0, glass2, ecoli4, glass4, abalone-9_vs_18, glass5, yeast-2_vs_8, yeast4, yeast5, yeast6, abalone19.

Imbalanced (SMOTE): [5-fcv] glass1, ecoli-0_vs_1, wisconsin, pima, iris0, glass0, yeast1, vehicle1, vehicle2, vehicle3, haberman, glass-0-1-2-3_vs_4-5-6, vehicle0, ecoli1, new-thyroid2, new-thyroid1, ecoli2, segment0, glass6, yeast3, ecoli3, page-blocks0, vowel0, glass2, ecoli4, glass4, abalone-9_vs_18, glass5, yeast-2_vs_8, yeast4, yeast5, yeast6, abalone19.

Results obtained: ZIP file
- XLS file CSV file - Standard results
- XLS file CSV file - Standard results (non-evolutionary)
- XLS file CSV file - Imbalanced results
- XLS file CSV file - Imbalanced results (non-evolutionary)
- XLS file CSV file - Imbalanced results (SMOTE)
- XLS file CSV file - Imbalanced results (non-evolutionary) (SMOTE)
- XLS file CSV file - Kappa results
- XLS file CSV file - Kappa results (non-evolutionary)