Which impurity measure is commonly used in CART and does not rely on logarithms?

Which impurity measure is commonly used in CART and does not rely on logarithms?

• A. Entropy
• B. Information gain
• C. Purity
• D. Gini coefficient

Answer: (D)

In CART, the impurity measure used for deciding where to split is chosen for being simple and fast to compute, and it should reflect how mixed the classes are in a node without needing any logarithms. The Gini impurity does exactly that with a straightforward formula: 1 minus the sum of the squared class probabilities in the node. If a node contains only one class, the Gini impurity is zero, meaning it’s completely pure; the more evenly the classes are represented, the higher the impurity. Entropy and information gain, by contrast, rely on logarithms of probabilities (H = -sum p log p, with information gain as the reduction in this entropy after a split). Those come from information-theoretic measures and are used in other algorithms like ID3 and C4.5, but they aren’t the default impurity measure in CART. So the impurity measure that is commonly used in CART and does not rely on logarithms is the Gini coefficient.

In CART, the impurity measure used for deciding where to split is chosen for being simple and fast to compute, and it should reflect how mixed the classes are in a node without needing any logarithms. The Gini impurity does exactly that with a straightforward formula: 1 minus the sum of the squared class probabilities in the node. If a node contains only one class, the Gini impurity is zero, meaning it’s completely pure; the more evenly the classes are represented, the higher the impurity.

Entropy and information gain, by contrast, rely on logarithms of probabilities (H = -sum p log p, with information gain as the reduction in this entropy after a split). Those come from information-theoretic measures and are used in other algorithms like ID3 and C4.5, but they aren’t the default impurity measure in CART.

So the impurity measure that is commonly used in CART and does not rely on logarithms is the Gini coefficient.