Using local node information in decision trees : coupling a local decision rule with an off-centered entropy

Nguyen-Khang Pham , Thanh Nghi Do , Philippe Lenca , Stéphane Lallich

Dealing with skewed class distribution and cost- sensitive data has been recognized as one of the 10 most challenging problems in data mining research. These problems have been reported to hinder the performance of classifiers, especially on the minority class. To deal with this problem in decision tree induction we proposed an off-centered entropy while other authors proposed an asymmetric entropy. Com- pared to Shannon's entropy both of them take their maximum value for a distribution fixed by the user instead of an uniform distribution. We here also propose to use in each leaf of the tree a local class labeling rule instead of the classical majority rule that mechanically favors the majority class i.e. the negative one. In this paper we briefly present the concepts of the three entropies and the new class labeling rule. This allows us to propose an adaptive learning of decision trees. We then compare their effectiveness on 25 imbalanced data sets. All our experiments are founded on the C4.5 decision tree algorithm, in which only the function of entropy and class labeling rule are modified. The results are promising and show the interest of our proposal.

Mots clés: Decision tree

Domaines: Ordinateur et société [cs.CY]

Nguyen-Khang Pham, Thanh Nghi Do, Philippe Lenca, Stéphane Lallich. Using local node information in decision trees : coupling a local decision rule with an off-centered entropy. International Conference on Data Mining 14-17 July 2008, Las Vegas, Nevada, USA, Jul 2008, Las Vegas, United States. pp.117 - 123. ⟨hal-02120810⟩